code-critic-model/Qwen3-8B-Critic-SFT-Detailed-Prompt

TEXT GENERATIONPricing:Input $0.468 / Output $1.82Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 9, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

Qwen3-8B-Critic-SFT-Detailed-Prompt is an 8 billion parameter critic model developed by Shubham Gandhi et al., trained on detailed teacher critiques for code agents. This model, based on the Qwen/Qwen3-8B architecture with a 32,768 token context length, is designed to evaluate and provide concrete code-level fixes. It serves as a comparison arm in prompt ablation studies, demonstrating how detailed critiques can impact critic model performance for code generation tasks.

Loading preview...

Qwen3-8B-Critic-SFT-Detailed-Prompt Overview

This model is an 8 billion parameter critic, part of the research presented in "Steer, Don't Solve: Training Small Critic Models for Large Code Agents" by Shubham Gandhi et al. It is specifically trained on detailed teacher critiques, which include concrete code-level fixes, distinguishing it from its counterpart, Qwen3-8B-Critic-SFT, which uses high-level critiques.

Key Characteristics & Training

  • Critic Functionality: Designed to evaluate code agent trajectories and provide specific, actionable feedback, often including code snippets.
  • Training Data: Fine-tuned on the code-critic-model/critic-sft-cwm-only-detailed-prompt dataset, comprising 3,135 examples. This data was generated by querying Claude Opus 4.6 with a detailed prompt for critiques on CWM-32B agent trajectories across R2E-Gym tasks.
  • Base Model: Built upon the Qwen/Qwen3-8B architecture.
  • Context Length: Supports a sequence length of 32,768 tokens.

Performance Insights

While trained on detailed critiques, the paper indicates that this critic model generally performs weaker than critics trained on high-level critiques in terms of resolve rate on SWE-bench Verified. For instance, when paired with Qwen3-Next-80B-A3B, it achieved a 24.6% resolve rate compared to 25.2% for the high-level critic. This suggests that while detailed critiques provide more explicit guidance, they may lead to a less effective critic model in certain scenarios, potentially due to agents copying code rather than learning to generate it independently.

Finetunes
1 models