enmingzhangzz/Qwen2.5-VL-7B-OPSD-VisionZip-r010-BTIP-Top10-ForwardKL-gap01-original10240

Qwen2.5-VL-7B-OPSD-VisionZip-r010-BTIP-Top10-ForwardKL-gap01-original10240

Final PEFT/LoRA adapter from an OPSD experiment.

Variant

• —Objective: B-TIP Soft-OR Top-K OPSD
• —Base model: Qwen/Qwen2.5-VL-7B-Instruct
• —Training samples: 10240
• —Dataset tag: openmmreasoner_llava_cot_exact_prefix10240_decontam_v1_seed42
• —Balanced base-outcome sampling: false
• —Pruning: visionzip
• —Vision-token retention ratios: [0.1]
• —EMA teacher decay: 0.9999
• —Global batch size: 32 (4 GPUs x micro-batch 8 x accumulation 1)
• —LoRA: r=16, alpha=32
• —Image pixels: 846720

• —B-TIP mode: token_btip_soft_or_topk
• —Budget axis: B_t = JSD(P_student,b || P_student,b+gap)
• —B/B+ intervention gap: 0.01 (absolute vision-token retention ratio)
• —Teacher/student divergence: forward KL
• —Soft-OR score: s_t = B_hat_t + delta_hat_t - B_hat_t * delta_hat_t
• —Deterministic Top-K token retention: 0.1
• —B-axis clipping quantile: 0.98

Files

adapter_model.safetensors and adapter_config.json are the final adapter at step 10240. Audit and reproducibility metadata are under training/.

Adapter SHA256: 83163543b80289e45758aa6ea5ef386304165ad01e62b21cbdb358d1458c2c66

Load this adapter on top of Qwen/Qwen2.5-VL-7B-Instruct with PEFT. The VisionZip runtime patch used by the OPSD repository is still required for pruned inference.