kingzcx/Vision-OPD-Qwen3.5-9B-Option-Tail-Ori-Template

Vision-OPD Qwen3.5-9B Option-Tail (Original Template)

This checkpoint is a Qwen3.5-9B Vision-OPD Option-Tail variant trained with option-space on-policy self-distillation on the Vision-OPD-6K dataset.

The student processes the original image while an EMA teacher processes the privileged region-focused image. The Option-Tail objective distills the probability distribution over legal answer-option tokens and a residual tail bucket containing all non-option probability mass.

This run uses the original Qwen3.5 perception chat template. Unlike the companion Option-Tail experiment, it does not rewrite the multiple-choice instruction to add the word directly. All other training settings are unchanged, making this a template ablation checkpoint.

Training configuration

• —Base model: Qwen/Qwen3.5-9B
• —Retained samples after prompt-length filtering: 6,215
• —Maximum prompt length: 5,120 tokens
• —Maximum response length: 1 token
• —Training epochs: 1
• —Optimizer steps: 64
• —Global training batch size: 96
• —Rollouts per prompt: 8
• —Learning rate: 2e-6
• —Distillation loss: JSD (alpha=0.5)
• —Teacher regularization: EMA
• —Teacher update rate: 0.05
• —Teacher image key: bbox_images

Training completed all 64 steps. No benchmark evaluation is included with this checkpoint.

Loading

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "kingzcx/Vision-OPD-Qwen3.5-9B-Option-Tail-Ori-Template"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

Citation

@article{yuan2026vision,
  title={Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation},
  author={Yuan, Qianhao and Lou, Jie and Yu, Xing and Lin, Hongyu and Sun, Le and Han, Xianpei and Lu, Yaojie},
  journal={arXiv preprint arXiv:2605.18740},
  year={2026}
}