Modelpublic

RidaKo/qwen3-vl-road-signs-lt-lora

sourceHugging Faceupdated 4mo agoView on Hugging Face

0likes9downloads

Model Card

Qwen3-VL Road Signs LT LoRA

This repository contains a LoRA adapter fine-tuned for Lithuanian road sign image captioning.

The adapter is intended to be used together with the base model unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit. It is not a standalone full model.

Model Details

—Base model: unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit
—Model type: Vision-language LoRA adapter
—Fine-tuning method: Supervised fine-tuning with LoRA
—Language: Lithuanian
—Task: Road sign image captioning
—Libraries: PEFT, Transformers, TRL, Unsloth

Intended Use

The model is intended to generate short Lithuanian captions for road sign images. The expected output should describe visible road signs, their type, meaning, readable text, numbers, arrows, and uncertainty when signs are unclear.

Example instruction:

text

Trumpai, 1–3 sakiniais, aprašyk nuotraukoje matomus kelio ženklus lietuviškai. Paminėk ženklo tipą, reikšmę, matomą tekstą, skaičius ar rodykles. Jei ženklas neaiškus, per toli arba tekstas neįskaitomas, taip ir parašyk. Nespėliok nematomų detalių.

Training Data

The model was fine-tuned on a custom dataset of Lithuanian road sign images with human-written Lithuanian captions.

Final dataset split:

Split	Images
Training	161
Validation	18
Test	50

The 50-image test set was kept separate and was not used during training.

Training Configuration

Parameter	Value
Epochs	4
Learning rate	5e-5
LoRA rank	32
LoRA alpha	32
LoRA dropout	0.05
Batch size per device	1
Gradient accumulation steps	4
Effective batch size	4
Max generated tokens	256

Evaluation

The baseline model and the fine-tuned model were evaluated on the same fixed 50-image test set.

Model	BLEU	chrF	ROUGE-L	Keyword F1	Visible Value F1
Baseline	0.339	28.810	0.130	0.097	0.131
Fine-tuned	10.669	45.230	0.318	0.538	0.438

The fine-tuned model generated shorter and more task-specific Lithuanian captions than the baseline model.

Limitations

The model can still confuse visually similar signs, misread or hallucinate numbers, and describe unclear signs too confidently. It should be treated as an experimental captioning adapter, not as a safety-critical road sign recognition system.

How to Load

python

from unsloth import FastVisionModel

base_model = "unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit"
adapter_model = "RidaKo/qwen3-vl-road-signs-lt-lora"

model, tokenizer = FastVisionModel.from_pretrained(
    base_model,
    load_in_4bit=True,
)

model.load_adapter(adapter_model)