Anakonkai/qwen3.5-9b-lora-traffic-rag-sft-v2

qwen3.5-9b-lora-traffic-rag-sft-v2 (ablation / negative result)

QLoRA adapter trained on (question + evidence_card) -> answer format as an ablation.

This adapter underperforms the canonical v2 and is kept only for reproducibility of the negative result.

Motivation

Investigate whether fine-tuning on the evidence-card format (instead of raw context) would help reduce the train/inference distribution gap.

Training

• —Base: Qwen/Qwen3.5-9B, 4-bit NF4 via unsloth.
• —LoRA rank 32, alpha 64.
• —1 epoch, lr 3e-5, batch 2, grad_accum 8.
• —v1: 90% evidence card / 10% no-context.
• —v2: 50% evidence card / 40% raw context / 10% no-context.

Result

Card-specialised SFT teaches the model to refuse when card fields are empty, raising false-refusal ~6-7x. The original LoRA-v2 generalises better.

Related repos

• —Canonical: Anakonkai/qwen3.5-9b-lora-traffic-v2
• —Code: github.com/Anakonkai01/nlp-traffic-laws