Modelpublic

vgandhi13/Qwen2.5-VL-7B-RLVR-ReasoningOnly-ThinkLite

sourceHugging Faceotherupdated 24d agoView on Hugging Face

0likes33downloads

Model Card

Qwen2.5-VL-7B — RLVR Reasoning-Only Baseline (ThinkLite-VL-70k)

GRPO finetune of Qwen/Qwen2.5-VL-7B-Instruct on ThinkLite-VL-70k, trained as the "Reasoning-only" baseline from Beyond Reasoning Gains: Mitigating General-Capability Forgetting in Large Reasoning Models (arXiv:2510.21978, Phan et al.), Table 2.

This is a reproduction of a baseline, not of the paper's proposed method (RECAP). Its purpose is to exhibit the failure mode the paper describes: reasoning-focused RLVR that improves the target task while degrading unrelated general capabilities.

Intended use

Research on capability forgetting in RLVR. It is a deliberately un-regularized baseline — no replay, no KL — so it is not recommended as a general-purpose VLM. The base model is the better choice for anything other than studying this effect.

Training


Base	`Qwen/Qwen2.5-VL-7B-Instruct`
Data	`russwang/ThinkLite-VL-70k` (69,497 rows after dropping 3 with empty ground truth)
Algorithm	GRPO, no reference KL (`use_kl_loss=False`, `use_kl_in_reward=False`)
Steps	500
Hardware	4 × A100-80GB
Effective batch	16 prompts × 4 rollouts (64 rollouts / optimizer step)
Optimizer	AdamW, β=(0.9, 0.999)
LR	1e-6, 10% warmup, cosine decay to 0
Precision	bf16 + FlashAttention
Framework	verl @ `c2429f29`

Reward = accuracy + thinking-format, each binary and equally weighted.

Results

Held-out validation (500 samples from ThinkLite-VL-70k, never trained on):

step	0 (base)	100	300	500
accuracy	0.288	0.784	0.816	0.814
format	0.788	0.996	1.000	1.000

Accuracy nearly triples. Format saturates by step ~50 and contributes no further gradient signal — the asymmetry between a quickly-saturating objective and a slowly-improving one is exactly what the paper's RECAP scheduler exploits.

Accuracy plateaus around step 250–300; the final 200 steps add little.

Known deviations from the paper

—8 → 4 GPUs. The paper uses 8-way data parallelism; this run uses 4 with 4 gradient accumulation steps instead of 2. The effective batch and optimizer math are identical.
—Cosine instead of linear LR decay. The paper decays linearly to 0. verl's FSDP path accepts only constant or cosine (verl/workers/config/optimizer.py), so cosine with min_lr_ratio=0 was used — same 10% warmup, same endpoint, different curve between.
—Reward weights. The paper specifies "fixed reward weights" for this baseline without giving values, and notes prior work doubles accuracy relative to format. This run uses 1:1. Under GRPO's group-normalized advantages a saturated format reward contributes no within-group variance, so after ~step 50 the weighting is largely inert.
—General-capability benchmarks are not yet evaluated here. The forgetting claim (e.g. LISA 65.13 → 57.58) is not verified in this card; only the in-domain reasoning metric above is measured.

Limitations

Inherits the base model's limitations plus the regressions this baseline is designed to produce. Do not deploy. One upstream verl bug was patched to complete training: the text-only-batch fallback in qwen2_vl.py called .mean() on a BaseModelOutputWithPooling returned by transformers ≥5.0; the fix routes it through verl's existing unpack_visual_output helper.

License

Derivative of Qwen2.5-VL-7B-Instruct and governed by the Qwen license.