CoolFace
Modelpublic

Anakonkai/qwen3.5-9b-lora-traffic-v2

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes10downloads
Model Card

qwen3.5-9b-lora-traffic-v2

QLoRA adapter fine-tuned on Qwen/Qwen3.5-9B for Vietnamese traffic-law QA.

This is the canonical adapter for configs C (no RAG) and D (with RAG).

Training

  • —Base: Qwen/Qwen3.5-9B, 4-bit NF4 via unsloth.
  • —LoRA rank 32, alpha 64, targeting all 7 modules (q/k/v/o, gate/up/down_proj).
  • —Data: data/splits_filtered/qa_train.jsonl (1762 pairs) + dev 220.
  • —Schedule: 2 epochs, lr 5e-5, batch 2, grad_accum 8 (eff batch 16), cosine warmup 5%.
  • —CONTEXT_KEEP_PROB=0.9 — 90% of samples keep the raw legal context in the prompt.

Usage

python
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Anakonkai/qwen3.5-9b-lora-traffic-v2",
    max_seq_length=2048, load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

Results (145-sample labelled eval)

  • —Config C (LoRA, no RAG): ROUGE-L 0.389, BLEU 0.146, BERTScore 0.638, LLM-Judge 0.39.
  • —Config D (LoRA + RAG): ROUGE-L 0.515, BLEU 0.350, BERTScore 0.692, LLM-Judge 0.69.

Related repos