Datasetpublic

dougalldeepmind/2026-08-06-qwen36-table2-80-self-reflection-20-10k-train-mixture

Qwen3.6 Table2 80% + SynthDoc self-reflection 20% — 10k-example training bundle field value experiment One-epoch Qwen3.6-27B assistant-only LoRA SFT (r64): Matthew's exact 7,999 Table-2 rows + 2,000 first-person self-reflection records — the self-reflection twin of LASR-Callum/2026-08-04-qwen36-lora-table2-synthdoc-rank-64, differing ONLY in the 20% slice (difficult-advice -> self-reflection). date_generated 2026-08-06 (mixture; Table-2 rows verbatim from the… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-06-qwen36-table2-80-self-reflection-20-10k-train-mixture.

sourceHugging Faceapache-2.0updated 27d agoView on Hugging Face

0likes164downloads

Dataset Card

Qwen3.6 Table2 80% + SynthDoc self-reflection 20% — 10k-example training bundle

field	value
`experiment`	One-epoch Qwen3.6-27B assistant-only LoRA SFT (r64): Matthew's exact 7,999 Table-2 rows + 2,000 first-person self-reflection records — the self-reflection twin of `LASR-Callum/2026-08-04-qwen36-lora-table2-synthdoc-rank-64`, differing ONLY in the 20% slice (difficult-advice -> self-reflection).
`date_generated`	2026-08-06 (mixture; Table-2 rows verbatim from the 2026-08-04 arm, self-reflection corpus 2026-08-03 + 2026-08-06 top-up)
`constitution`	`constitutions/claude_distilled_12_principles_mid/constitution.md` in `source_repo` (the self-reflection corpus's target; since 2026-08-05 byte-identical to the 9-principle generation-time snapshot). Table-2 rows connect to none.
`source_repo`	`teaching_claude_why_replication` @ `6e5be8bc61800cfbb4ea02f2145bba2dc624bac9` (branch `model-eval-model-data-gen`, uncommitted mixture/train configs included in `code.tar.gz`)
`models`	Base `Qwen/Qwen3.6-27B`. Corpus generators: `anthropic/claude-haiku-4.5` + `anthropic/claude-sonnet-5` via OpenRouter (see corpus repo).
`generation_config`	No generation in this step: deterministic seed-0 shuffle and exact example-count fill (7,999 + 2,000).
`schema`	`mixture.jsonl`: `text` = Qwen3.6 ChatML with think blocks preserved; `source` = `table2` or `self_reflection`; `n_tokens`.
`provenance`	`uv run python scripts/data/mixture/build_mixture.py --config configs/data/mixture/2026-08-07_qwen36_table2_self_reflection_20_80.yaml`; Table-2 side extracted verbatim by `scratch/prep_table2_from_matthew.py` from `LASR-Callum/2026-08-04-table2-synthdoc-h200x4-train` (minus its 2,203 difficult-advice rows); self-reflection side is `data/self_reflection_sft_all.jsonl` = `LASR-Callum/2026-08-03-synthdoc-self-reflection` (592) + pilot run 20260806114324 (18) + top-up run 20260806115149 (~1,410).

Mixture

json

{
  "total": {
    "examples": 9999,
    "tokens": 9741068
  },
  "by_source": {
    "table2": {
      "examples": 7999,
      "tokens": 4295425,
      "share_pct_examples": 80.0,
      "share_pct_tokens": 44.1
    },
    "self_reflection": {
      "examples": 2000,
      "tokens": 5445643,
      "share_pct_examples": 20.0,
      "share_pct_tokens": 55.9
    }
  },
  "mixture_path": "output/mixture_table2_selfreflect_20_80/20260806_150300/mixture.jsonl"
}

Mixture SHA-256: c8f291c639e5a559f5aa77d3cfb267ee200cf2f1daf33c11e80de45508636336.

Training

code.tar.gz + configs/train/2026-08-07_lora_qwen36_table2_self_reflection_rank64.yaml: BF16 LoRA r=64/alpha=128, 1 epoch, batch 1 x grad-accum 16 (global 16), lr 1e-4 cosine, 5% warmup, weight decay 0.01, maxseqlen 8192, assistant-only loss with the generation-boundary think rule (empty markers masked, real traces supervised). Launched by scripts/gpu/runpod_train.py on a credential-free pod; adapter pushed to LASR-Callum/qwen3.6-27b-lora-table2-selfreflect-r64 from the driver machine.