Datasetpublic

dougalldeepmind/2026-08-31-difficult-advice-716-seeds-bundle

da716 seed replicates — training bundle (seeds 42 and 69) code.tar.gz (trainer + src/ + the two seed configs) beside seed 0's mixture, byte-identical. scripts/gpu/runpod_train.py up reads both from this one repo. field value experiment Seed replicates of the da716 arm (Table2 9,284 filtered + difficult-advice-v2 716, 7.16%) so the arm carries training-seed variance like its siblings. da716 was the last arm on a single seed and is the comparison baseline for the… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-31-difficult-advice-716-seeds-bundle.

sourceHugging Faceapache-2.0updated 26d agoView on Hugging Face

0likes88downloads

Dataset Card

da716 seed replicates — training bundle (seeds 42 and 69)

code.tar.gz (trainer + src/ + the two seed configs) beside seed 0's mixture, byte-identical. scripts/gpu/runpod_train.py up reads both from this one repo.

field	value
`experiment`	Seed replicates of the da716 arm (Table2 9,284 filtered + difficult-advice-v2 716, 7.16%) so the arm carries training-seed variance like its siblings. da716 was the last arm on a single seed and is the comparison baseline for the generator sweep.
`date_generated`	2026-08-31
`constitution`	`claude_distilled_12_principles_mid` (9 principles) — the difficult-advice half is generated from it. Source corpus: `LASR-Callum/2026-08-13-haiku45-sonnet45-difficult-advice-diversity-gated-voice-linted`
`source_repo`	https://github.com/Matthew-Bozoukov/Lessonsfromconstituitional_AFT @ `274b2de`
`models`	Trains `Qwen/Qwen3.6-27B`. The mixture's difficult-advice half was written by `anthropic/claude-haiku-4.5` (drafts) + `anthropic/claude-sonnet-5` (rewrite) via OpenRouter — see the source corpus card.
`generation_config`	No sampling here. Seeds 42 and 69; every other field identical to seed 0 (LoRA r64/alpha128, global batch 16, lr 1e-4 cosine, warmup 0.05, maxseqlen 8192, dynamic batching, 1 epoch, 2xH200 DDP).
`schema`	`t2_9284_da716_10k.jsonl` — JSONL, `text` (pre-rendered to the Qwen3.6 chat template) + `source`. 10,000 rows = 716 difficult-advice + 9,284 Table2.
`provenance`	`scratch/da716_seeds/prepare_bundle.py` then `scratch/publish_train_bundle.py --repo LASR-Callum/2026-08-31-difficult-advice-716-seeds-bundle --train_config configs/train/2026-08-31_lora_qwen36_t2_9284_da716_dynbatch_s42_2xh200.yaml --extra configs/train/2026-08-31_lora_qwen36_t2_9284_da716_dynbatch_s69_2xh200.yaml`; launch with `scripts/gpu/runpod_train.py up --bundle LASR-Callum/2026-08-31-difficult-advice-716-seeds-bundle --gpu_count 2`.

Mixture provenance

t2_9284_da716_10k.jsonl is copied VERBATIM from `LASR-Callum/2026-08-14-table2-9284-difficult-advice-716-train` — sha256 bc5c18e0fae1823ca7bd1b5bdae58e585a833958c632e21014b640733959b41e, asserted by the staging script. It is never rebuilt: the mixture builder's shuffle depends on the corpus it reads, so rebuilding would reorder every row and make these runs differ from seed 0 in their data as well as their seed.

Deviation from seed 0, stated plainly

Seed 0 (2026-08-14, commit 40ed848) ran a trainer that read a LOCAL data_path and whose main() took no overrides, so it cannot be driven by the current pod launcher at all. These replicates therefore run the CURRENT trainer. The two differences that touch training were checked before launching: build_labels gained an optional mask_spans argument that is inert when unused (this arm does not use it), and warmup_ratio now passes through a compatibility shim that emits the identical schedule when SFTConfig still accepts the field and the converted equivalent when it does not. Everything else in the diff is plumbing (data_repo/data_file, push, stamps, logging).