OpenReviewer: A Specialized LLM for Scientific Peer Reviews
Llama-OpenReviewer-8B is an 8 billion parameter language model developed by maxidl, specifically engineered to generate high-quality, critical reviews for machine learning and AI conference papers. This model is the core component of the OpenReviewer system, designed to assist in the academic peer-review process.
Key Capabilities and Training
- Specialized Fine-tuning: The model was fine-tuned on approximately 79,000 high-confidence expert reviews from 32,000 individual papers sourced from OpenReview, covering major conferences like ICLR and NeurIPS.
- Base Model: It is built upon Llama-3.1-8B-Instruct and was full fine-tuned using
axolotl. - Context Length: Supports a substantial context length of 128k tokens (though the model card states 32768 tokens, the README mentions 128k for training).
- Review Generation: Trained to follow specific reviewer guidelines and generate structured reviews in markdown format, including sections like Summary, Soundness, Presentation, Contribution, Strengths, Weaknesses, Questions, and an overall Rating.
- System Prompt Adherence: Utilizes a detailed system prompt that guides the model to act as an expert AI conference reviewer, emphasizing comprehensive evaluation, identification of strong/weak points, and constructive feedback.
Intended Use and Considerations
- Primary Use Case: Generating detailed and critical peer reviews for scientific papers in AI and machine learning.
- Transparency: Users are explicitly asked to disclose the model's use when employing it for official peer-reviewing tasks.
- Input Format: Expects the full paper text, preferably in markdown format, as input for review generation.