vuhaian/qlora_27b_53k_phase1_adapter

vuhaian/qlora27b53kphase1adapter

QLoRA adapter for Qwen/Qwen3.8-27B — phase1 of a two-phase curriculum.

• —phase 1: vuhaian/53k_lastdance, 80 steps, lr 5e-5 cosine
• —phase 2: vuhaian/top3_lastdance, 80 steps, lr 3e-5, continuing phase 1's adapter
• —r=32, alpha=64, 400 modules (48 linear-attn x3, 16 full-attn x4, 64 MLP x3), 217.6M params
• —base quantised NF4 for training (4-bit reaches 95.3% of this dense model)
• —packed to 16,384 tokens, global batch 16, loss on the last assistant turn only

Held-out eval (both splits removed before phase 1):

Scale note: 80 steps is ~4% of a phase-1 epoch and ~24% of a phase-2 epoch. Load with peft.PeftModel.from_pretrained on top of the base.