Modelpublic

xiaorui638/Qwen3-VL-8B-ZwZ-S57-verify

sourceHugging Faceapache-2.0updated 29d agoView on Hugging Face

0likes21downloads

Model Card

Qwen3-VL-8B-Instruct — ZwZ S57K verify (step 149)

GRPO-trained on 57,447 false-premise twin pairs (positive/negative, 2:1). Each negative asks about something absent from the image; the model must pick a premise-denial option rather than answer. The model emits explicit <check>…</check><verdict>yes|no</verdict> steps before its answer.

Reward

gated_mult: acc * (0.5 + 0.5 * coverage) + 0.1 * format, where coverage is the fraction of gold sub-questions covered by the model's emitted checks, scored by an LLM judge (Qwen3-30B-A3B-Instruct-2507). 150 steps, rollout 384.

Results

metric	base	ZwZ-8B	plain GRPO	this model
ZTB-mcq paired (in-domain)	11.20	7.22	39.34	43.83
VERVE paired (in-domain)	72.93	45.67	79.68	78.39
DASH-B	75.28	58.65	84.08	86.84
DASH-B acc_no	54.36	19.09	77.48	82.03
mcq-OOD(10) mean	65.35	68.34	65.88	66.44

Against ZwZ-8B (same base, trained on 74K pure positives): ZwZ leads the visual-search cluster but is worse than the untrained base on every discriminative axis — VERVE paired 45.67 vs base 72.93, DASH-B acc_no 19.09 vs 54.36. An all-positive recipe cannot refuse a false premise.

Inference

Use the verify v2.1 prompt. Under the default prompt in-domain paired drops to 30.53 — the checking behaviour is prompt-bound, not internalised.

Caveats

Single seed. mme-realworld-lite is 8–10 points below base for every trained arm here (uninvestigated denial-distractor over-transfer).