bihungba1101/ntkt-conversation-vietnamese-full-text-qwen3.5-0.8b-sft

Qwen3.5 0.8B NTKT Vietnamese Conversation SFT

This PEFT LoRA adapter fine-tunes Qwen/Qwen3.5-0.8B for conversational knowledge tracing. It predicts Correct or Incorrect from a learner's Vietnamese interaction history represented as full text.

Training

• —Method: supervised fine-tuning with assistant-only loss
• —Context length: 15,000 tokens
• —Steps: 100
• —Effective batch size: 8
• —LoRA rank/alpha: 16/16
• —Trainable parameters: 6,389,760 (0.74%)
• —Final training loss: 0.3562

View the training run on Weights & Biases.

Evaluation

Evaluation used a learner-disjoint validation split with 115,853 labeled predictions across 2,361 conversations.

Usage

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

repo_id = "bihungba1101/ntkt-conversation-vietnamese-full-text-qwen3.5-0.8b-sft"
model = AutoPeftModelForCausalLM.from_pretrained(repo_id, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(repo_id)

The adapter expects the included chat template and the same full-text conversation representation used during training.

Framework versions

• —PEFT 0.18.1
• —TRL 0.23.1
• —Transformers 5.2.0
• —PyTorch 2.10.0+cu128
• —Datasets 4.3.0
• —Tokenizers 0.22.2