code-critic-model/Qwen3-4B-Critic-SFT-DPO
The Qwen3-4B-Critic-SFT-DPO model, developed by code-critic-model, is a 4 billion parameter critic model fine-tuned with Direct Preference Optimization (DPO) for steering large code agents. Starting from Qwen3-4B-Critic-SFT, it excels at providing structured critiques, detecting error categories, and suggesting recovery actions for coding agents. This model is specifically designed to improve the resolve rate of various coding agents by offering guidance rather than directly solving problems, making it a specialized tool for enhancing automated code development workflows.
Loading preview...
Overview
This model, Qwen3-4B-Critic-SFT-DPO, is a 4 billion parameter critic model developed by code-critic-model. It is an evolution of the Qwen3-4B-Critic-SFT model, further enhanced through Direct Preference Optimization (DPO). Its primary function is to act as a 'critic' alongside a frozen coding agent, providing structured feedback and guidance at various steps of the agent's trajectory. This approach, detailed in the paper "Steer, Don't Solve: Training Small Critic Models for Large Code Agents," focuses on steering the agent rather than directly generating code patches.
Key Capabilities & Training
- Structured Critiques: The model reads the agent's trajectory and outputs short, structured critiques including error categories, evidence, recovery actions, task status, and overall guidance.
- DPO Training: It was trained using DPO on 1,409 preference pairs, where critiques from the SFT critic were judged by Claude Opus 4.6 for correctness and clarity, with the best critiques marked as 'chosen' and worst as 'rejected'.
- Performance Improvement: Benchmarks on SWE-bench Verified show that integrating this DPO critic significantly improves the resolve rate across various coding agents (e.g., Qwen3-32B, GPT-OSS-20B, GLM-4.7-Flash-30B-A3B), often outperforming its SFT-only predecessor and even larger 8B SFT critics.
Use Cases
This model is ideal for developers and researchers working on:
- Enhancing Code Agents: Integrating a feedback loop to improve the performance and efficiency of automated code generation and debugging systems.
- Automated Software Development: Utilizing a small, specialized model to guide larger, more general-purpose coding agents.
- Research in AI Feedback Systems: Exploring the impact of critic models on agent behavior and task completion rates in complex environments like SWE-bench.