Modelpublic

xiaorui638/qwen3vl-8b-s85k-plain-grpo

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes15downloads

Model Card

Qwen3-VL-8B-Instruct — GRPO on false-premise twin pairs (s85k, plain)

Qwen/Qwen3-VL-8B-Instruct further trained with GRPO (plain accuracy reward, no verification shaping) on an 85k-example set of false-premise twin pairs: each image carries a positive question answerable from the image and a negative twin whose premise is false, where the correct response is an explicit premise-denial option. Pos:neg ratio 2:1.

The goal is a model that rejects false premises without becoming a blanket denier — so the headline metric is paired accuracy (both twins of a pair correct), not per-question accuracy.

Results — ZoomBench/TreeBench MCQ (in-domain)

2,343 questions = 584 positives + 1,759 negative/positive twin pairs. vLLM, seed 42, T=0.7, bf16; judge = rule-match then Qwen3-30B-A3B-Instruct. Single seed.

Paired accuracy — negative and its positive twin both correct

category	pairs	Qwen3-VL-8B base	this model
needle_obj	445	20.00	54.83
needle_attr	393	12.21	48.35
ref1obj	269	12.27	46.84
ref1attr	226	6.19	39.38
ref1rel	202	3.47	24.26
ref2obj	84	5.95	42.86
ref2attr	74	1.35	31.08
ref2rel	66	0.00	13.64
micro (all pairs)	1759	11.20	43.55

Per-question accuracy

category	n	Qwen3-VL-8B base	this model
positive	584	44.52	58.56
needle_obj	445	53.71	92.13
needle_attr	393	27.74	87.28
ref1obj	269	33.09	86.62
ref1attr	226	18.58	79.20
ref1rel	202	13.86	50.50
ref2obj	84	22.62	82.14
ref2attr	74	9.46	71.62
ref2rel	66	6.06	33.33
overall	2343	34.02	74.82

Read the paired column, not the overall. Per-question accuracy on this benchmark rewards denial: a model that denies every premise scores well on the 1,759 negatives while failing all 584 positives. The positive half is this model's actual bottleneck — positive accuracy moves only 44.52 → 58.56, and most remaining paired error is a wrong positive twin, not a missed denial. Relational negatives (ref_*_rel) stay hard.

This benchmark is in-domain — built by the same pipeline as the training data — so it measures how well the target behaviour was learned, not general ability transfer.

Usage

Weights are bfloat16.

python

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model = Qwen3VLForConditionalGeneration.from_pretrained(REPO_ID, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(REPO_ID)

vLLM:

bash

vllm serve REPO_ID --dtype bfloat16 --limit-mm-per-prompt image=1

The model was trained with a think-then-answer format prompt; it emits reasoning followed by the answer in <answer>...</answer> tags.

Training


Base	`Qwen/Qwen3-VL-8B-Instruct`
Algorithm	GRPO (KL disabled), rollout n=8 @ T=1.0
Reward	accuracy only (letter/string match, else LLM judge)
Data	85,000 false-premise twin examples, 2:1 pos:neg
Batch	rollout 384 / global 96, lr 1e-6, 1 epoch
Checkpoint	global step 221 (final)
Framework	EasyR1 (veRL), 24× GH200

Limitations

—Single seed; no variance estimate. Treat <1 pt differences as noise.
—Evaluated in-domain here. It trades some general MCQ ability for premise robustness; consult the fuller evaluation before using it as a general-purpose VLM.
—Inherits all limitations of the base model.