dougalldeepmind/2026-08-02-qwen36-synthdoc-package-mixture-10-90
Qwen3.6-27B SFT mixture — synthdoc_v2 10/90 10% difficult-advice / 90% TULU3 replay, 996,193 tokens total. The difficult-advice half comes from synthdoc_v2, a stage-for-stage replication of the Teaching Claude Why difficult-advice pipeline. Source Examples Tokens Share difficult-advice (synthdoc_v2) 58 99,847 10.02% TULU3 replay 1,402 896,346 89.98% Total 1,460 996,193 md5 16b8876b9c480ca9e0351ebe92a516f2. How the difficult-advice data was made… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-02-qwen36-synthdoc-package-mixture-10-90.
Qwen3.6-27B SFT mixture — synthdoc_v2 10/90
10% difficult-advice / 90% TULU3 replay, 996,193 tokens total. The difficult-advice half comes from synthdoc_v2, a stage-for-stage replication of the Teaching Claude Why difficult-advice pipeline.
md5 16b8876b9c480ca9e0351ebe92a516f2.
How the difficult-advice data was made
Six separate stages, each with its own model and its own cached snapshot:
Trait-balanced. The constitution splits into 8 numbered principles, and the source pool (difficult_advice_pool.jsonl) holds exactly 25 examples for each, so no principle dominates. Each record carries its target principle in metadata.trait_id/trait_name.
Think-block convention
Zero rows carry an empty <think></think>, which is the documented pattern that trains a model to stop reasoning. Re-rendering from messages will not reproduce these strings.
Intended use
Assistant-only loss: mask everything outside an assistant turn. ~78% of tokens are supervised under that scheme.