Datasetpublic

dougalldeepmind/2026-08-31-difficult-advice-principle-scoped-702-seeds-bundle

chunk-only 702 seed replicates — training bundle (seeds 42 and 69) code.tar.gz (trainer + src/ + the two seed configs) beside seed 0's mixture, byte-identical. scripts/gpu/runpod_train.py up reads both from this one repo. field value experiment Seed replicates so this arm carries training-seed variance. Table2 9,284 filtered + chunk-only difficult advice 702 (7.03%). The rewrite stages never saw the constitution, only their one target principle. Between-seed spread on… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-31-difficult-advice-principle-scoped-702-seeds-bundle.

sourceHugging Faceapache-2.0updated 27d agoView on Hugging Face

0likes68downloads

Dataset Card

chunk-only 702 seed replicates — training bundle (seeds 42 and 69)

code.tar.gz (trainer + src/ + the two seed configs) beside seed 0's mixture, byte-identical. scripts/gpu/runpod_train.py up reads both from this one repo.

field	value
`experiment`	Seed replicates so this arm carries training-seed variance. Table2 9,284 filtered + chunk-only difficult advice 702 (7.03%). The rewrite stages never saw the constitution, only their one target principle. Between-seed spread on ODCV is 1.2–9.4 pp, so a single-seed arm cannot be ranked against its siblings at all.
`date_generated`	2026-08-31
`constitution`	`claude_distilled_12_principles_mid` (9 principles). Source corpus: `LASR-Callum/2026-08-13-difficult-advice-v2`
`source_repo`	https://github.com/Matthew-Bozoukov/Lessonsfromconstituitional_AFT @ `c3942f4`
`models`	Trains `Qwen/Qwen3.6-27B`. The difficult-advice half was written by `anthropic/claude-haiku-4.5` (scenarios, drafts) + `anthropic/claude-sonnet-5` (rewrites) via OpenRouter — see the source corpus card.
`generation_config`	No sampling here. Seeds 42 and 69; every other field identical to seed 0 (LoRA r64/alpha128, global batch 16, lr 1e-4 cosine, warmup 0.05, maxseqlen 8192, dynamic batching, 1 epoch, 2×H200 DDP).
`schema`	`t2_9284_da_chunk_only_702.jsonl` — JSONL, `text` (pre-rendered to the Qwen3.6 chat template) + `source`. 9,986 rows.
`provenance`	`scratch/da716_seeds/prepare_bundle.py --arm chunk_only` then `scratch/publish_train_bundle.py --repo LASR-Callum/2026-08-31-da-chunk-only-702-seeds-bundle --train_config configs/train/2026-08-31_lora_qwen36_t2_9284_da_chunk_only_702_dynbatch_s42_2xh200.yaml --extra configs/train/2026-08-31_lora_qwen36_t2_9284_da_chunk_only_702_dynbatch_s69_2xh200.yaml`; launch with `scripts/gpu/runpod_train.py up --bundle LASR-Callum/2026-08-31-da-chunk-only-702-seeds-bundle --gpu "NVIDIA H200" --gpu_count 2 --mixture t2_9284_da_chunk_only_702.jsonl`.

Mixture provenance

t2_9284_da_chunk_only_702.jsonl is copied VERBATIM from `LASR-Callum/2026-08-21-table2-9284-da-chunk-only-702-train` — sha256 52182e2e13294141e6c408850ec9e7ffd756873b005bf203c6d5ec3fe7a96bcb, asserted by the staging script along with the 9,986-row count. It is never rebuilt: the mixture builder's shuffle depends on the corpus it reads, so rebuilding would reorder every row and make these runs differ from seed 0 in their data as well as their seed.

Deviation from seed 0, stated plainly

Seed 0's trainer read a LOCAL data_path and its main() took no overrides, so it cannot be driven by the current pod launcher at all. These replicates therefore run the CURRENT trainer. The two differences that touch training were checked rather than assumed: build_labels gained an optional mask_spans argument that is inert when unused (this arm does not use it), and warmup_ratio now passes through a compatibility shim that emits the identical schedule when SFTConfig still accepts the field and the converted equivalent when it does not. Everything else in the diff is plumbing (data_repo/data_file, push, stamps, logging).