dougalldeepmind/2026-08-08-table2-9000-synthdoc-1000-trait-balanced-len-8000-train-mixture
Table-2 (9,000) + synthdoc difficult-advice (1,000, trait-balanced), all rows <= 8,000 tokens 10,000-example SFT mixture for Qwen3.6-27B. Train on mixture_think.jsonl — every assistant turn carries a think block, which the trainer's preserve-thinking gate requires. field value experiment 90/10-by-examples SFT mixture: 9,000 spec-filtered Table-2 instruction rows + 1,000 difficult-advice documents drawn evenly across all 9 constitution traits date_generated… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-08-table2-9000-synthdoc-1000-trait-balanced-len-8000-train-mixture.
Table-2 (9,000) + synthdoc difficult-advice (1,000, trait-balanced), all rows <= 8,000 tokens
10,000-example SFT mixture for Qwen3.6-27B. Train on `mixture_think.jsonl` — every assistant turn carries a think block, which the trainer's preserve-thinking gate requires.
Composition
Difficult-advice rows per trait — all 9, as even as 1,000 allows: {'t1': 112, 't2': 111, 't3': 111, 't4': 111, 't5': 111, 't6': 111, 't7': 111, 't8': 111, 't9': 111}
Think blocks, and one repair worth knowing about
- Table-2 rows carry an empty
<think></think>marker as inference-time CONTEXT; training masks the whole marker out of the loss, because teaching a model to emit it is the documented reasoning-collapse pattern for Qwen3.x. - Difficult-advice rows carry a real reasoning trace and are supervised.
The upstream Table-2 export marked only each row's FINAL assistant turn, leaving 362 bare turns in multi-turn rows — enough to fail the trainer's preserve-thinking gate outright. Those are repaired here. Verified census over this file: 10,354 assistant turns = 1,000 real traces + 9,354 empty markers + 0 bare.
Supervised-token accounting
Measured with the trainer's own build_labels, not estimated:
10% of examples is 40% of the loss. Difficult-advice rows average ~1,653 rendered tokens against Table-2's ~511, and 80% of each is supervised (real traces) versus 41% for Table-2. Read the mixture ratio in supervised tokens, not example counts.
Longest row 7,997 / 8000 tokens; 0 rows dropped for length.
The 8,000-token cap is applied to the POOL, not the selection
Every candidate row was rendered, marker-repaired and measured BEFORE sampling, and only fitting rows entered the pool: 9,249 of 9,284 Table-2 rows fit (35 excluded); 2,203 of 2,203 difficult-advice rows fit (0 excluded). Gating after selection would instead have dropped whichever long rows happened to be picked, silently breaking both the exact 9,000 / 1,000 counts and the even trait split. Verified on the published file: max row 7,997 tokens, 0 rows over 8,000.
The 35 excluded rows are almost all LongAlign (206 -> 173 in the selection), which is the only long-context source in the Table-2 mixture.
Status
Not yet trained on, not evaluated.