xiaorui638/Qwen3.5-9B-ZwZ-S57-verify-anneal

Qwen3.5-9B — ZwZ S57K verify→plain curriculum (step 149)

GRPO-trained on 57,447 false-premise twin pairs (positive/negative, 2:1), where each negative asks about something not in the image and the model must pick a premise-denial option instead of answering.

The recipe

Reward is gated_mult: acc * (alpha + (1-alpha) * coverage), where coverage is how many gold sub-questions the model's own emitted <check> questions cover, scored by an LLM judge. The curriculum retires that scaffold: alpha 0.5→1.0 and the format weight 0.1→0 linearly over steps 40–80, so the last ~70 steps are pure outcome-only GRPO from a verify-shaped policy.

Motivation: coverage saturates (~0.95 by step 60), after which the scaffold adds no gradient but keeps rehearsing a ~75%-yes verdict distribution that degrades out-of-domain denial calibration. Annealing it away fixed that.

Results (vs the same base and the un-annealed arm)

53.55 paired is the best in-domain result in the project. The un-annealed arm's OOD yes-bias (DASH-B acc_no 52.35) is largely repaired.

Inference

Use the verify v2.1 prompt it was trained with — under the default prompt in-domain paired drops to ~42.6. Needs vLLM >= 0.18 (model_type=qwen3_5); weights are already in the stock HF key layout (model.visual.*).

Caveats

Single seed. Trained for in-domain false-premise robustness: general-ability aggregates are at or slightly below the plain-GRPO arm (mcq-mean 70.97 vs 71.68), and mme-realworld-lite carries a persistent ~4-point tax vs base.