dougalldeepmind/2026-08-08-table2-9000-synthdoc-1000-trait-balanced-train-mixture
Table-2 (9,000) + synthdoc difficult-advice (1,000, trait-balanced) 10,000-example SFT mixture for Qwen3.6-27B. Train on mixture_think.jsonl — every assistant turn carries a think block, which the trainer's preserve-thinking gate requires. field value experiment 90/10-by-examples SFT mixture: 9,000 spec-filtered Table-2 instruction rows + 1,000 difficult-advice documents drawn evenly across all 9 constitution traits date_generated 2026-08-08 constitution… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-08-table2-9000-synthdoc-1000-trait-balanced-train-mixture.
Table-2 (9,000) + synthdoc difficult-advice (1,000, trait-balanced)
10,000-example SFT mixture for Qwen3.6-27B. Train on `mixture_think.jsonl` — every assistant turn carries a think block, which the trainer's preserve-thinking gate requires.
Composition
Difficult-advice rows per trait — all 9, as even as 1,000 allows: {'t1': 112, 't2': 111, 't3': 111, 't4': 111, 't5': 111, 't6': 111, 't7': 111, 't8': 111, 't9': 111}
Think blocks, and one repair worth knowing about
- Table-2 rows carry an empty
<think></think>marker as inference-time CONTEXT; training masks the whole marker out of the loss, because teaching a model to emit it is the documented reasoning-collapse pattern for Qwen3.x. - Difficult-advice rows carry a real reasoning trace and are supervised.
The upstream Table-2 export marked only each row's FINAL assistant turn, leaving 362 bare turns in multi-turn rows — enough to fail the trainer's preserve-thinking gate outright. Those are repaired here. Verified census over this file: 10,354 assistant turns = 1,000 real traces + 9,354 empty markers + 0 bare.
Supervised-token accounting
Measured with the trainer's own build_labels, not estimated:
10% of examples is 40% of the loss. Difficult-advice rows average ~1,650 rendered tokens against Table-2's ~539, and 80% of each is supervised (real traces) versus 41% for Table-2. Read the mixture ratio in supervised tokens, not example counts.
Longest row 8,191 / 8192 tokens; 0 rows dropped for length.
Status
Not yet trained on, not evaluated.