code-critic-model/Qwen3-8B-Critic-SFT
Qwen3-8B-Critic-SFT is an 8 billion parameter model developed by code-critic-model, fine-tuned from Qwen3-8B. It functions as a critic model designed to steer large code agents by providing structured critiques, error detection, and recovery actions based on agent trajectories. Trained on critiques generated by Claude Opus 4.6, this model excels at improving the resolve rate of various coding agents on benchmarks like SWE-bench Verified, particularly for Python-based tasks, with a context length of 32,768 tokens.
Loading preview...
Qwen3-8B-Critic-SFT: A Specialized Critic for Code Agents
Qwen3-8B-Critic-SFT is an 8 billion parameter model, fine-tuned from Qwen3-8B, developed by code-critic-model. Its primary function is to act as a critic model that works alongside frozen coding agents. Instead of solving problems directly, it reads agent trajectories and provides structured critiques including error categories, evidence, recovery actions, and overall guidance.
Key Capabilities & Training:
- Agent Steering: Designed to steer coding agents by providing timely feedback every few steps, improving their problem-solving efficiency.
- Critique Generation: Generates short, structured critiques based on observed agent behavior, identifying errors and suggesting high-level recovery strategies.
- Teacher-Student Training: Fine-tuned on 6,447 examples of critiques written by Claude Opus 4.6, distilling its expertise into a smaller, specialized model.
- Context Length: Supports a sequence length of 32,768 tokens, allowing it to process extensive agent trajectories.
- Python-focused: Training data primarily involved Python repositories and trajectories in the mini-swe-agent format.
Performance & Use Cases:
This critic model significantly improves the resolve rate of various coding agents on benchmarks like SWE-bench Verified. For instance, it boosted Qwen3-Next-80B-A3B's resolve rate from 20.0% to 25.2% and GLM-4.7-Flash-30B-A3B's from 21.6% to 37.6% on SWE-bench Verified. It is intended to be integrated into agent workflows, typically served via vLLM, to provide real-time, step-aware feedback.
Limitations:
The model's training was exclusively on Python repositories and mini-swe-agent formatted trajectories. Its critiques are reliable within the distribution of its teacher's training data and it is evaluated as a critic, not a standalone code generator.