dougalldeepmind/2026-08-04-sft-mixture-table2-8000-plus-synthdoc-2203

SFT mixture — Table 2 instruction-tuning (8,000, filtered) + difficult advice (2,203)

10,203 examples. Combines a spec-filtered reproduction of the paper's Table 2 instruction-tuning mixture with the full difficult-advice corpus generated against a 9-principle distilled constitution.

Composition

The two halves weigh very differently depending on which token you count. Difficult advice is 21.6% of examples and 46.0% of rendered tokens, but 62.6% of supervised tokens — it is long-form assistant prose with reasoning traces, whereas much of the Table 2 half is long context with short answers. Weight by supervised tokens, not example counts.

By source

longalign alone is 20% of rendered tokens and 0.7% of supervised: long documents in, one-line answers out. apigen_function_calling is similar (schema-heavy prompts, short tool calls).

Think blocks — three states, kept distinct

• —synthdoc rows carry a real `<think>` trace and are fully supervised. This is the ingredient the difficult-advice result depends on: the assistant deliberates about the value at stake before answering.
• —Table 2 rows carry no think block at all — neutral, training the model neither to emit nor to suppress reasoning.
• —No row carries an empty `<think></think>`. Training a model to emit that marker is the documented reasoning-collapse pattern for Qwen3.x. If you want the empty marker as inference-time context on the non-reasoning half, add it with src/data/add_empty_think_multi.py and set mask_empty_think: true so its tokens are excluded from the loss.

Filtering (Table 2 half)

Judged by openai/gpt-5.6-terra against the full constitution, per the paper's recipe. 9,285 of 10,000 kept (7.15% rejected), then 8,000 sampled stratified by source.

Read the caveat before using this as a spec filter: rejections were ~30 toxic and ~685 spec_violation, with zero wrong_identity or no_self_disclaimer — the two failure modes the paper names. About half the spec_violation rejections are instruction-following failures (a summary ignoring its three-sentence limit), which is a real data-quality problem but a broader screen than the paper describes. smol_summarize lost 31% on that basis. Per-sample categories are in the parent repo's verdicts.jsonl if you want to rebuild with a narrower rule.

Parent datasets

• —difficult advice: `2026-08-04-synthdoc-difficult-advice-9-principles`
• —Table 2 sources: No Robots, Tulu3 SFT (IF personas), smoltalk (numina-cot-100k, self-oss-instruct, smol-constraints, apigen-80k, smol-summarize, longalign), LIMA

Caveats

• —LIMA came from `64bits/lima_vicuna_format`, a third-party re-upload: GAIR/lima ships a loading script the Hub no longer supports. Verify provenance if that matters to you.
• —LongAlign is not representative. Its rows average 10,677 tokens; only the shortest ~18% fit under the 8192 cap, so 965 were dropped to find 216.
• —79 of 10,000 judge replies did not parse and were kept rather than dropped — filtering on unparseable output would silently delete good data.
• —Not evaluated: no model has been trained on this mixture yet.

Usage

from datasets import load_dataset
ds = load_dataset("LASR-Callum/2026-08-04-sft-mixture-table2-8000-plus-synthdoc-2203", data_files="mixture.jsonl", split="train")

Rows are pre-rendered with the Qwen3.6 chat template. Train on text directly and build the assistant-only loss mask from the rendered string (src/train/masking.py) rather than re-rendering from messages.