dougalldeepmind/2026-07-31-qwen36-sft-mixture-10-90-assistant-loss-only
Qwen3.6-27B SFT mixture — 10-90_assistant_loss_only 10% difficult-advice / 90% TULU3 replay, by token. Built for training with loss on assistant tokens only. mixture.jsonl is byte-identical (md5 af628722652f05debf5cffd44db09f88, 2,257 rows) to the mixture used by the full-token arm …-tulu-lora-10-90, so the loss mask is the only difference between the two runs. Source Rows Tokens Share Supervised difficult-advice 147 149,816 10.0% 85.55% TULU3 replay 2,110 1,343,608… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-07-31-qwen36-sft-mixture-10-90-assistant-loss-only.
Qwen3.6-27B SFT mixture — 10-90assistantloss_only
10% difficult-advice / 90% TULU3 replay, by token. Built for training with loss on assistant tokens only.
mixture.jsonl is byte-identical (md5 af628722652f05debf5cffd44db09f88, 2,257 rows) to the mixture used by the full-token arm `…-tulu-lora-10-90`, so the loss mask is the only difference between the two runs.
Files
Reasoning traces
Zero rows carry an empty `<think></think>` (asserted at build time). That distinction is the point: an empty think block is Qwen3.6's explicit do-not-deliberate marker and trains a model to stop reasoning, whereas absent markup says nothing either way. The builder gets there by appending a throwaway user turn — which demotes the assistant turn from final to historical, so the template takes its no-think branch — then cutting that turn back off.
Re-rendering these conversations from messages will not reproduce the training data.
What "assistantlossonly" means
Every token outside an assistant turn is -100 and contributes no loss. A supervised span starts immediately after the <|im_start|>assistant\n header — which the model is given at inference and never has to produce — and ends after the closing <|im_end|>, which it must produce in order to stop.
<|im_start|> MASKED
assistant MASKED
\n MASKED
<think> LOSS <- supervision starts at the first generated tokenassistant_spans.jsonl records those spans as character offsets into text, so the mask is reproducible without our code.
TRL's `assistant_only_loss` flag cannot do this on Qwen3.6. It requires {% generation %} markers the template lacks, and it re-renders from messages, discarding the think-block convention above. The spans are derived from the rendered text via the fast tokenizer's offset mapping instead.
Provenance
src/experiments/build_mixture.py with configs/mixture_qwen36_10_90.yaml, seed 0, from `allenai/tulu-3-sft-mixture` and matboz/difficult-advice-qwen3. Replay conversations over 2,048 Qwen3.6 tokens were dropped rather than truncated.