dougalldeepmind/2026-08-02-qwen36-lora-500k-da20-numina

Qwen3.6-27B — 500k, 20% difficult-advice + maths-weighted remainder

LoRA adapter trained on 499,344 tokens with loss on assistant tokens only, empty-think markers excluded from the loss, for 1 epoch at lr 4e-5.

Training data: `qwen3.6-27b-mixture-500k-da20-numina`.

Within the non-difficult-advice 80.8%: NuminaMath 66.8%, TULU3 + No Robots 33.2%.

Trait balance

The difficult-advice half holds exactly 7 examples for each of the constitution's 8 principles. That quantisation is why its share lands at 19.19% rather than exactly 20%: 7 per trait gives 95,813 tokens, 8 gives 109,109 (21.8%). Exact trait balance was preferred over an exact 20%.

Think-block convention

Three different treatments, one per data type:

The empty marker is Qwen3.6's non-thinking marker, placed exactly where apply_chat_template puts it. It is masked from the loss: the model is conditioned on it but never trained to emit one, since learning to emit an empty think block is the documented reasoning-collapse pattern. NuminaMath is left unmarked because marking it "non-thinking" would contradict its own chain-of-thought content.

Training

Verified before training, on the box: zero empty-think markers inside any supervised span, zero user or system tokens in the loss, and all 56 difficult-advice rows retaining their real reasoning traces.

loss_type: nll is set because TRL's default chunked-CE path patches the LM head and reads forward.__func__, which fails on this checkpoint's functools.partial forward. The loss is mathematically the same.

Related runs

Loss rises with the difficult-advice share because open-ended advice with reasoning traces is a harder next-token target than mathematical solutions. That is a property of the data, not of model quality.

Not yet evaluated on ODCV-Bench or agentic-misalignment.

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/2026-08-02-qwen36-lora-500k-da20-numina")
model = model.merge_and_unload()

Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language checkpoint.