code-critic-model/Qwen3-8B-Critic-SFT-Qwen-only
Qwen3-8B-Critic-SFT-Qwen-only is an 8 billion parameter critic model developed by Shubham Gandhi, Yiqing Xie, Atharva Naik, Ruichen Zhu, and Carolyn Rose, designed to provide structured critiques for coding agents. Trained exclusively on Qwen3-Next-80B-A3B trajectories, this model identifies error categories, suggests recovery actions, and offers overall guidance without generating code patches. With a 32,768 token context length, it specializes in improving the performance of large language models on code-related tasks by acting as an external feedback mechanism.
Loading preview...
Qwen3-8B-Critic-SFT-Qwen-only: A Specialized Code Critic Model
This model is an 8 billion parameter critic, part of the research presented in "Steer, Don't Solve: Training Small Critic Models for Large Code Agents." Unlike traditional code generation models, its primary function is to analyze a coding agent's trajectory and provide structured feedback rather than writing code itself.
Key Capabilities
- Structured Critiques: Reads a coding agent's trajectory and outputs detected error categories, evidence, a recovery action, task status, and overall guidance.
- Targeted Training: Specifically trained on critiques of Qwen3-Next-80B-A3B trajectories, making it highly specialized for this agent's outputs.
- Performance Enhancement: Demonstrated to improve the resolve rate of coding agents on benchmarks like SWE-bench Verified. For instance, it boosted Qwen3-Next-80B-A3B's resolve rate from 20.0% to 26.2%.
- High Context Length: Supports a sequence length of 32,768 tokens, allowing for comprehensive analysis of longer code trajectories.
Training Details
The model was fine-tuned using LLaMA-Factory on the critic-sft-qwen-only dataset, comprising 1,915 examples. This dataset was generated from 483 R2E-Gym instances, with Qwen3-Next-80B-A3B producing the trajectories and Claude Opus 4.6 acting as the teacher for critiques. The training utilized a qwen3_nothink chat template and focused loss on the final critique turn.
When to Use This Model
This critic is particularly effective when paired with Qwen3-Next-80B-A3B agents for code generation tasks, where its specialized training leads to optimal performance. For broader applicability across different coding agents, the Qwen3-8B-Critic-SFT model, trained on a mixed corpus, is recommended as a more general-purpose critic.