experiment | Arm: train the 702 principle-scoped difficult-advice rows on their VISIBLE ANSWER ONLY -- the reasoning trace stays in the token stream as unsupervised context (no truncation, full forward pass) and simply earns no loss, while the 9,284 Table2 rows train exactly as in the control. The EXACT COMPLEMENT of the CoT-only arm on the same base: on every one of the 702 rows, supervised(cot) + supervised(answer) == supervised(control), verified token-for-token (420,037 + 401,033 = 821,070, 0 mismatches). That partition is what makes the two arms comparable to EACH OTHER, not merely to their shared control. |
date_generated | 2026-08-31 |
constitution | claudedistilled07principlesapproved (constitutions/claudedistilled07principlesapproved/constitution.md) |
source_repo | Matthew-Bozoukov/teachingclaudewhy_replication @ bd2fb80866942e338a5d1cf62f1f9defaf33be2c |
models | token stream Qwen/Qwen3.6-27B (tokenizer + ModelProfile literals) |
generation_config | none — no model is sampled here. The build is a deterministic per-row field addition over a pinned control. |
schema | text (rendered Qwen3.6 chat, IDENTICAL to the control mixture); source (mixture source name); supervise ('answer' on the 702 difficultadvicechunk_only rows, absent elsewhere = 'all') |
provenance | uv run python scratch/cotonly/buildmixture.py --repo LASR-Callum/2026-08-21-table2-9284-difficult-advice-principle-scoped-702-train-mixture --file t29284dachunkonly702.jsonl --revision fa98fadeee72 --dasource difficultadvicechunkonly --nda 702 --nrows 9986 ; verified by --mode answer ; verified by uv run python scratch/cotonly/verifymixture.py --arm chunkonly --mode answer (text byte-identical to the control on every row, mask gate passed on BOTH supervise modes with the real Qwen3.6 tokenizer, stratified sample 64 all + 64 cot, 0 truncated) |
control_dataset | LASR-Callum/2026-08-21-table2-9284-difficult-advice-principle-scoped-702-train-mixture (t29284dachunkonly_702.jsonl @ fa98fadeee72) - same text, no supervise column |
trained_control | LASR-Callum/2026-08-21-qwen36-lora-table2-9284-difficult-advice-chunk-only-702-rank-64-dynbatch - ODCV 11.5% [6.2, 19.6], severity 0.62 (2 passes, 65 cells). Trained on exactly this control mixture, so the arm differs from it in the supervise field alone. |
rows | 9986 (702 carry supervise=cot) |
intervention_scale | at maxseqlen 8192: forward tokens 6,033,864 -> 6,033,864 (-0.0% overall, -0.0% on the difficult-advice rows); supervised tokens 2,867,910 -> 2,447,873 (-14.65%); difficult-advice share of the training signal 28.6% -> 16.4% |
usage | src/train/trainlora.py consumes `supervise` via buildlabels; a trainer that ignores the column silently trains the control instead. Consumed by configs/train/loraqwen36t29284synthdoc716cotonlydynbatch2xh200.yaml; the control is the sibling config without _cotonly. |
caveat | seqmeantokenmeanloss weights each EXAMPLE at 1/global_batch, so this arm concentrates the same per-example weight onto the trace rather than reducing the rows' influence — roughly doubling the per-CoT-token gradient weight. It is 'reasoning only, at double density', not 'the control minus its answer term'. |
