HqH1111/Qwen3-VL-2B-NuScenes-DrivingQA

Qwen3-VL-2B — nuScenes Driving QA + Counterfactual (LoRA, merged)

LoRA adaptation of Qwen3-VL-2B-Instruct for front-camera driving visual question answering on nuScenes, combining NuScenes-QA (perception: existence / count / status / position) and OmniDrive reasoning + counterfactual QA ("if you follow trajectory X, what happens?"). Vision encoder frozen; LoRA on the LLM + aligner, then merged into a standalone checkpoint. Trained on driving data only (no general-domain replay).

Results (front camera, greedy decoding, vs stock Qwen3-VL-2B)

General-ability retention: MMStar 43.2→44.8, AI2D 72.2→73.2, ScienceQA 74.4→80.0, ChartQA 80.4→76.8, RealWorldQA 68.2→63.2 (largely preserved; several improve).

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
m = AutoModelForImageTextToText.from_pretrained("HqH1111/Qwen3-VL-2B-NuScenes-DrivingQA", torch_dtype="auto", device_map="auto")
p = AutoProcessor.from_pretrained("HqH1111/Qwen3-VL-2B-NuScenes-DrivingQA")

Training: Qwen3-VL-2B-Instruct · ViT frozen · LoRA r64/α128 (all-linear) · bf16 · 3×A100 · ms-swift · 50k samples (NuScenes-QA 28k + OmniDrive 22k) · 2 epochs. Non-commercial research use (nuScenes / OmniDrive terms).