code-critic-model/Qwen3-8B-Critic-SFT-Detailed-Prompt

Qwen3-8B-Critic-SFT-Detailed-Prompt

An 8B critic from Steer, Don't Solve: Training Small Critic Models for Large Code Agents, trained on teacher critiques collected with the detailed prompt instead of the high-level one. It is the comparison arm of the prompt ablation (Table 4). The critic trained on high-level critiques is Qwen3-8B-Critic-SFT.

The two prompts differ in what the teacher is allowed to say. The high-level prompt asks for error detection plus short guidance and forbids full solutions. The detailed prompt lets the teacher propose concrete code-level fixes, so its critiques are longer and often contain code. The paper shows that agents copy this code (Table 5), and that a small critic trained on detailed critiques ends up weaker than one trained on high-level critiques.

All released models and datasets are listed on the organization page. Code and configs are in the critic-training repository.

Where it appears in the paper

Original run name: qwen3-8b-full-sft-prm-r2egym-swebench-k5-opus-distill-32k-lr5e6-multiturn. In the repository the detailed prompt is the prm_issue_res config family and the high-level prompt is prm_issue_res_instructions.

Training data

code-critic-model/critic-sft-cwm-only-detailed-prompt, 3,135 examples.

• —Tasks: 500 R2E-Gym instances from matplotlib, moto, and sympy, disjoint from SWE-bench Verified.
• —Agent that produced the trajectories: CWM-32B, 500 trajectories.
• —Teacher: Claude Opus 4.6, queried every 5 agent steps with the detailed prompt.

The same 500 tasks and agent as critic-sft-cwm-only; only the teacher prompt differs. Fewer examples survive the 32K token limit because detailed critiques are longer.

Training setup

Identical to Qwen3-8B-Critic-SFT apart from the data. Full-parameter SFT with LLaMA-Factory, config finetuning/qwen3_8b_critic_full_sft_l40s_train_multiturn_resumable.yaml.

Results

Resolve rate on SWE-bench Verified, from Table 4 of the paper. Both critics were run with the same high-level, step-aware inference prompt; only the training critiques differ.

Note that the high-level critic was trained on the larger CWM plus Qwen3-Next corpus, so this comparison also reflects the corpus difference.

How to use

Same serving and launch procedure as Qwen3-8B-Critic-SFT: serve with vLLM in bf16 and pass the served name to scripts/run_critic_max150.sh with --prm. The served name must have an entry in mini-swe-agent/configs/litellm_model_registry.json; add one for this model if you use a new name.

Citation

@misc{gandhi2026steerdontsolvetraining,
  title={Steer, Don't Solve: Training Small Critic Models for Large Code Agents},
  author={Shubham Gandhi and Yiqing Xie and Atharva Naik and Ruichen Zhu and Carolyn Rose},
  year={2026},
  eprint={2606.21811},
  archivePrefix={arXiv},
  primaryClass={cs.SE},
  url={https://arxiv.org/abs/2606.21811}
}