Datasetpublic

dougalldeepmind/2026-08-31-difficult-advice-716-cot-only-supervision

CoT-only supervision mixture, difficult-advice-v2 (Table2 9,284 + DA-v2 716) field value experiment Arm: train the 716 difficult-advice rows on their REASONING ONLY — each row is truncated at its </think> close, so the visible answer leaves both the loss and the forward pass — while the 9,284 Table2 rows train exactly as in the control. Tests whether the difficult-advice effect on agentic misalignment is carried by the reasoning or by the answer. date_generated… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-31-difficult-advice-716-cot-only-supervision.

sourceHugging Faceupdated 26d agoView on Hugging Face

0likes52downloads

Dataset Card

CoT-only supervision mixture, difficult-advice-v2 (Table2 9,284 + DA-v2 716)

field	value
`experiment`	Arm: train the 716 difficult-advice rows on their REASONING ONLY — each row is truncated at its `</think>` close, so the visible answer leaves both the loss and the forward pass — while the 9,284 Table2 rows train exactly as in the control. Tests whether the difficult-advice effect on agentic misalignment is carried by the reasoning or by the answer.
`date_generated`	2026-08-31
`constitution`	claudedistilled07principlesapproved (constitutions/claudedistilled07principlesapproved/constitution.md)
`source_repo`	Matthew-Bozoukov/teachingclaudewhy_replication @ 967461aee5c1e4eaf239137202d14ef7c80371b8
`models`	token stream Qwen/Qwen3.6-27B (tokenizer + ModelProfile literals)
`generation_config`	none — no model is sampled here. The build is a deterministic per-row field addition over a pinned control.
`schema`	text (rendered Qwen3.6 chat, IDENTICAL to the control mixture); source (mixture source name); supervise ('cot' on the 716 difficult-advice rows, absent elsewhere = 'all')
`provenance`	uv run python scratch/cotonly/buildmixture.py --repo LASR-Callum/2026-08-14-table2-9284-difficult-advice-716-train --file t29284da71610k.jsonl --revision 89e80f9093c8 --dasource difficultadvicev2 ; verified by uv run python scratch/cotonly/verifymixture.py --arm da_v2 (text byte-identical to the control on all 10,000 rows, mask gate passed on BOTH supervise modes with the real Qwen3.6 tokenizer, stratified sample 64 all + 64 cot, 0 truncated)
`control_dataset`	LASR-Callum/2026-08-14-table2-9284-difficult-advice-716-train (t29284da716_10k.jsonl @ 89e80f9093c8) - same text, no supervise column
`trained_control`	LASR-Callum/2026-08-14-qwen36-lora-table2-9284-difficult-advice-716-rank-64-dynbatch (ODCV 16.3%, 2xH200 dynamic batching, seed 0) - the adapter trained on exactly this control mixture, so the arm differs from it in the `supervise` field alone.
`rows`	10000 (716 carry supervise=cot)
`intervention_scale`	at maxseqlen 8192: forward tokens 6,045,214 -> 5,633,597 (-6.81% overall, -39.81% on the difficult-advice rows); supervised tokens 2,878,904 -> 2,468,003 (-14.27%); difficult-advice share of the training signal 28.9% -> 17.1%
`usage`	src/train/trainlora.py consumes `supervise` via buildlabels; a trainer that ignores the column silently trains the control instead. Consumed by configs/train/2026-08-31loraqwen36table29284synthdoc716cotonlydynbatch.yaml; the control is the sibling config without `_cotonly`.
`caveat`	seqmeantokenmeanloss weights each EXAMPLE at 1/global_batch, so this arm concentrates the same per-example weight onto the trace rather than reducing the rows' influence — roughly doubling the per-CoT-token gradient weight. It is 'reasoning only, at double density', not 'the control minus its answer term'.