RRRayen/RoadScene-VLM-Qwen2.5-VL-3B-QLoRA

RoadScene-VLM QLoRA Adapter

This repository contains a QLoRA adapter for:

Qwen/Qwen2.5-VL-3B-Instruct

It was fine-tuned on a small BDD100K road-scene pilot dataset for structured multimodal road-scene understanding.

Task

Given a road image, the model outputs strict JSON containing:

• —weather
• —time of day
• —scene type
• —cars
• —buses
• —trucks
• —pedestrians
• —traffic lights
• —traffic signs

Example output:

{ "weather": "overcast", "timeofday": "daytime", "scene": "city street", "cars": 12, "buses": 0, "trucks": 1, "pedestrians": 2, "trafficlights": 3, "trafficsigns": 4 }

Training Configuration

• —Base model: Qwen/Qwen2.5-VL-3B-Instruct
• —Train / Validation / Test: 800 / 100 / 200
• —Epochs: 1
• —Quantization: 4-bit NF4
• —Double quantization: enabled
• —Compute dtype: BF16
• —LoRA rank: 16
• —LoRA alpha: 32
• —LoRA dropout: 0.05
• —Target modules: qproj, kproj, vproj, oproj
• —Learning rate: 1e-4
• —Effective batch size: 8
• —GPU: NVIDIA A100

Frozen Test Results

Key Findings

QLoRA substantially improved road-weather label alignment, strict JSON output compliance, and vehicle-count prediction.

For vehicle counting:

• —MAE decreased from 7.00 to 3.14
• —Spearman correlation increased from 0.517 to 0.764
• —prediction bias changed from -7.00 to +0.48

The QLoRA model also outperformed image-independent mean and median count baselines, suggesting that the improvement was not explained only by learning the dataset's average vehicle count.

Limitations

This is a small pilot experiment rather than a production autonomous-driving model.

The dataset is strongly imbalanced, and several rare classes have insufficient representation in the natural test set.

Scene classification showed increased majority-class bias toward city street.

The visual backbone was not directly fine-tuned.

Only one primary QLoRA configuration was evaluated, so the experiment does not establish an optimal hyperparameter setting.

Base Model

Qwen/Qwen2.5-VL-3B-Instruct

Please review the upstream model repository and license terms before reuse.