mjusdda/lab21-2A202601343-qwen35-triage-vi

Qwen3.5-4B LoRA — Vietnamese customer-support triage

LoRA adapter fine-tuned by Đỗ Ngọc Anh (2A202601343) for Lab 21 of the VinUniversity AICB program. It converts a Vietnamese customer-support ticket into one JSON object containing intent, urgency, product, and sentiment.

This repository contains an adapter, not a standalone model. Load it on top of `unsloth/Qwen3.5-4B`.

Evaluation

Full evaluation used 50 target samples and 15 general-regression samples. Generation was greedy on a Colab NVIDIA T4.

The adapter improves target score by +0.205 over the optimized-prompt baseline, but regression drops by -0.080, beyond the allowed tolerance of -0.020. The formal verdict is therefore FAILED. This is an honest experimental result, not a claim that the adapter is production-ready. See `submission/REPORT.md` and the machine-readable files under `results/`.

Training configuration

• —Base model: unsloth/Qwen3.5-4B
• —Method: PEFT LoRA, text-decoder linear layers
• —Rank / alpha: r=16, alpha=32
• —Learning rate: 1e-4
• —Precision: fp16
• —Epochs / optimizer steps: 2 / 30
• —Trainable parameters: 32,464,896
• —Peak allocated VRAM: 12.01 GB
• —Training time: 987.6 seconds
• —Loss mask: assistant-only; prompt tokens are excluded

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "unsloth/Qwen3.5-4B"
adapter_id = "mjusdda/lab21-2A202601343-qwen35-triage-vi"

tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    trust_remote_code=True,
    dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

ticket = "Shop ơi, đơn bàn phím cơ của tôi giao chậm quá. Nhờ kiểm tra gấp."
messages = [
    {"role": "system", "content": "Phân loại ticket sau."},
    {"role": "user", "content": ticket},
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=160, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

Intended use

This adapter is intended for education, reproducibility, and experiments with Vietnamese ticket triage. Expected closed-vocabulary values are:

• —intent: doi_tra, van_chuyen, hoan_tien, san_pham_loi, hoi_thong_tin
• —urgency: cao, trung_binh, thap
• —sentiment: tieu_cuc, trung_tinh, tich_cuc
• —product: product name copied from the ticket

Limitations

• —The training corpus contains only 225 training tickets and is synthetic.
• —General regression fell from 0.7578 to 0.6778, indicating measurable specialization or forgetting.
• —Six of 50 target examples scored 0.75; the recurring error was predicting urgency=trung_binh when the correct value was thap.
• —Do not deploy as a general assistant or use for consequential customer decisions without broader data, replay training, and independent evaluation.
• —No inference provider is required for this submission; the adapter must be loaded with its base model.

Reproducibility and files

• —Code and report: GitHub repository
• —Full report: `submission/REPORT.md`
• —Metrics: `results/verdict.json`, `results/runs.csv`, and `results/autopsy.json`

License

The adapter follows the Apache-2.0 license declared by the base model. The accompanying lab code repository is separately licensed under MIT.