brikdavies/msm-judged-steering-vectors

Judged difference-of-means steering vectors (MSM checkpoints)

Raw mean(pos) − mean(neg) residual-stream deltas for the Model Spec Midtraining checkpoints (arXiv 2605.02087, Llama-3.1-8B + chloeli LoRA adapters), built from how the BASELINE model actually behaves on each preference eval rather than from synthetic contrast pairs.

Method

1. 1.Generate. The baseline checkpoint free-answers every eval prompt (affordability 497, america 400, cheese 288) at Llama-3.1's shipped sampling defaults: temperature 0.6, topp 0.9, repetitionpenalty 1.0, seed 0, max_tokens 256. No answer forcing of any kind.
2. 2.Judge. A blind LLM judge (Claude Sonnet agents; chunks carry only question, displayed options, response — never gold labels or model identity) decides which displayed option each answer preferred. label = 1 iff the judge's choice is the value-aligned option. Null verdicts (no clear preference) are dropped; judge confidence is kept per record.
3. 3.Capture. The judged prompt+completion texts are teacher-forced through ALL 6 checkpoints (same chat-template tokenization as generation), capturing the mean over completion tokens (completion_mean) at all 33 residual streams (embed + 32 layers). bf16 forward, fp32 stored.
4. 4.Vector. Per (checkpoint, domain): raw per-stream mean(pos) − mean(neg) — [33, 4096], NO unit normalization (normalization: "raw_mean_delta").

Judged partitions (decided rows)

Layout

manifests/    judged_{domain}.jsonl   one record per decided row (prompt,
              completion, label, judge choice + confidence, sampling);
              manifest_stats.json pins the splits above
activations/  {variant}_judged_{domain}.safetensors  [n, 33, 4096] fp32
              completion_mean activations (+ _records.jsonl sidecars,
              row-aligned with the manifests)
vectors/      {domain}/{variant}/{vector.safetensors, metadata.json,
              README.md}  18 bundles; index.json has norms, sha256s, and
              mid-layer cosines vs baseline

Checkpoint keys: baseline, aft_cheese, msm_afford, msm_afford_aft, msm_america, msm_america_aft (adapters chloeli/llama-3.1-8b-* on meta-llama/Llama-3.1-8B).

Built by src/msm_eval/judged_vectors/ in midtraining_generalization; every bundle's metadata.json carries source sha256s, npos/nneg, and the judge protocol for provenance.