nanoswe/qwen-3.6-35b-a3b-teacher

Qwen3.6-35B-A3B Teacher

A lightly fine-tuned variant of Qwen/Qwen3.6-35B-A3B, trained to be a better generator of agentic software-engineering trajectories. We use it as the teacher for distillation experiments: it produces the SWE-bench-style trajectory data that smaller student models are then trained on.

The fine-tune is deliberately light — one epoch at LR 6e-6 — because the goal is to sharpen the base model's trajectory format and success rate, not to move it off its pretrained distribution.

Training data

Self-distillation on the base model's own rollouts. Qwen3.6-35B-A3B was sampled on SWE-smith task instances, each trajectory was graded by the SWE-smith harness, and the surviving set was curated as follows:

• —Correct only — the trajectory resolves its instance under recovered grading.
• —No cheating — trajectories whose assistant turns match a cheat-pattern regex are dropped.
• —No easy instances — instances solved by more than 75% of attempts (with at least 4 attempts) are dropped entirely, so the model does not spend its budget on problems it already solves.
• —Top-3 shortest per instance — among the correct trajectories for a base instance, keep the three with the fewest tokens, which favors direct solutions over meandering ones.
• —≤ 32,768 tokens.

The result is 14,268 training trajectories.

Training

Full-parameter bf16 SFT, one epoch:

Usage

Serve it as you would the base model. Two things differ:

• —Use the bundled `chat_template.jinja`. It emits plain <|im_start|>role\ncontent<|im_end|> turns with no <think>/</think> injection, which is what the training trajectories look like.
• —Thinking is off. The trajectories were generated with enable_thinking=False, and the model was fine-tuned on them in that form.

Weights are bf16. Our own trajectory generation runs it under vLLM with online FP8 quantization.

License

Apache 2.0, inherited from the base model.