yuanqianhao/Vision-OPD-4B

Vision-OPD-4B

<p align="center"> 📄 <a href="https://arxiv.org/abs/2605.18740">Paper</a> &nbsp;|&nbsp; 💻 <a href="https://github.com/VisionOPD/Vision-OPD">Code</a> &nbsp;|&nbsp; 🤗 <a href="https://huggingface.co/datasets/yuanqianhao/Vision-OPD-6K">Training Data</a> &nbsp;|&nbsp; 📦 <a href="https://hf.co/collections/yuanqianhao/vision-opd">Collection</a> </p>

Model Summary

Vision-OPD-4B is a multimodal LLM built on `Qwen/Qwen3.5-4B` and trained with Vision-OPD (Vision On-Policy Distillation), the regional-to-global self-distillation framework introduced in Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation.

MLLMs often answer fine-grained questions correctly when given an evidence-centered crop, yet fail on the corresponding full image — a persistent regional-to-global perception gap. Vision-OPD closes this gap by letting the model act as its own teacher: a crop-conditioned teacher and a full-image-conditioned student are instantiated from the same MLLM, the student generates on-policy rollouts, and training minimizes the token-level divergence between the two distributions along these rollouts. The benefit of visual zooming is thereby internalized into a single forward pass — no external teacher, no ground-truth labels, no reward verifiers, and no inference-time tool use.

With only 6.2K synthetic samples, Vision-OPD-4B substantially improves over its Qwen3.5-4B base and outperforms much larger open-source models (e.g., Kimi-K2.6, GLM-4.6V), closed-source models (e.g., GPT-5.4, GPT-5.2), and "Thinking-with-Images" agentic methods on fine-grained visual understanding benchmarks, while preserving general multimodal abilities on holdout tasks. See also the larger Vision-OPD-9B.

Key Features

• —🔍 Sees fine details in one forward pass — no cropping tools, zoom-in calls, or multi-step agentic inference at test time.
• —🪞 Self-distillation, label-free — the model's own crop-conditioned perception supervises its full-image policy; no external teacher, ground-truth labels, or reward verifiers.
• —🎯 On-policy + dense supervision — token-level divergence (JSD, β = 0.5) on the student's own rollouts avoids the exposure bias of SFT and the sparse rewards of RLVR.
• —📉 Data-efficient — trained on only 6.2K automatically synthesized samples (Vision-OPD-6K), one epoch.
• —🧠 No forgetting — general multimodal capability is preserved on holdout tasks (MMVP, CV-Bench, MMStar, POPE).

How It Works

1. 1.Data synthesis — evidence regions are proposed on unlabeled images via object identification and segmentation; a question answerable from the crop alone is generated, and the region's bounding box is overlaid on the full image for grounding. This yields triplets (full image, crop, question).
2. 2.Two policies, one model — the teacher conditions on the privileged crop, the student on the full image.
3. 3.On-policy distillation — the student samples a rollout (up to 1024 tokens); at every position both policies are evaluated on the same student prefix, and the per-token JSD between them is minimized (top-K = 100 logits distillation). Gradients flow only through the student.
4. 4.EMA teacher regularization — the teacher is updated as an exponential moving average of the student (α = 0.05), which prevents teacher–student co-adaptation and training collapse.

Quickstart

Weights load with standard transformers (≥ 5.5):

from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "yuanqianhao/Vision-OPD-4B"
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "url": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg"},
        {"type": "text", "text": "What is the color of the label on the bottle in the background?"},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Or serve with vLLM (OpenAI-compatible API), as used in our evaluation:

vllm serve yuanqianhao/Vision-OPD-4B --port 8000

To reproduce the reported numbers, use the released evaluation harness:

git clone https://github.com/VisionOPD/Vision-OPD && cd Vision-OPD
bash eval/run_eval.sh   # see README for API base / judge / benchmark env vars

Training Details

Training code: github.com/VisionOPD/Vision-OPD.

Citation

@article{yuan2026vision,
  title={Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation},
  author={Yuan, Qianhao and Lou, Jie and Yu, Xing and Lin, Hongyu and Sun, Le and Han, Xianpei and Lu, Yaojie},
  journal={arXiv preprint arXiv:2605.18740},
  year={2026}
}

License

Released under Apache-2.0, following the base model `Qwen/Qwen3.5-4B`.