Datasetpublic

dougalldeepmind/2026-08-02-qwen36-mixture-500k-numina-heavy

Qwen3.6-27B SFT mixture — 500k tokens, maths-weighted 498,035 tokens across 1,001 conversations, weighted toward maths: NuminaMath-CoT ~67%, TULU3 and No Robots sharing the remaining ~33%. md5 5d4d51e3e48c32d103ddf60930b3b0ca. Source Examples Tokens Share no_robots 271 82,239 16.51% numinamath_cot 611 333,351 66.93% tulu3 119 82,445 16.55% Total 1001 498,035 Sources: allenai/tulu-3-sft-mixture, AI-MO/NuminaMath-CoT, HuggingFaceH4/no_robots. Example counts… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-02-qwen36-mixture-500k-numina-heavy.

sourceHugging Faceodc-byupdated 1mo agoView on Hugging Face

0likes93downloads

Dataset Card

Qwen3.6-27B SFT mixture — 500k tokens, maths-weighted

498,035 tokens across 1,001 conversations, weighted toward maths: NuminaMath-CoT ~67%, TULU3 and No Robots sharing the remaining ~33%. md5 5d4d51e3e48c32d103ddf60930b3b0ca.

Source	Examples	Tokens	Share
no_robots	271	82,239	16.51%
numinamath_cot	611	333,351	66.93%
tulu3	119	82,445	16.55%
Total	1001	498,035

Sources: `allenai/tulu-3-sft-mixture`, `AI-MO/NuminaMath-CoT`, `HuggingFaceH4/no_robots`.

Example counts differ per source because the corpora differ in length: No Robots conversations are short, NuminaMath solutions are long.

Format

Every row is a pre-rendered Qwen3.6 chat string with no `<think>` block at all — not even an empty one, which is the documented pattern that trains a model to stop reasoning. The builder appends a throwaway user turn so the template takes its no-think branch, then cuts that turn back off. Re-rendering from messages will not reproduce these strings.

Conversations longer than 3,072 tokens were dropped rather than truncated mid-answer.

Intended use

Assistant-only loss: mask everything outside an assistant turn. 78.5% of tokens carry loss under that scheme.

Provenance

src/data/build_hf_mixture.py with configs/mixture_500k_numina_heavy.yaml, seed 0.