Datasetpublic

dougalldeepmind/2026-08-20-odcv-feature-discovery-difficult-advice-716-5-pct-vs-numina-control

LLM-driven feature discovery over ODCV-Bench rollouts from TWO matched Qwen3.6-27B LoRA arms — 9,284 filtered instruction rows plus 716 rows that differ only in kind (constitution-grounded difficult advice vs NuminaMath chain-of-thought) — asking which reasoning and action properties separate the two models, and which go with the judged misalignment. field value experiment LLM-driven feature discovery over ODCV-Bench rollouts from TWO matched Qwen3.6-27B LoRA arms — 9… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-20-odcv-feature-discovery-difficult-advice-716-5-pct-vs-numina-control.

sourceHugging Faceupdated 27d agoView on Hugging Face

0likes351downloads

Dataset Card

LLM-driven feature discovery over ODCV-Bench rollouts from TWO matched Qwen3.6-27B LoRA arms — 9,284 filtered instruction rows plus 716 rows that differ only in kind (constitution-grounded difficult advice vs NuminaMath chain-of-thought) — asking which reasoning and action properties separate the two models, and which go with the judged misalignment.

field	value
`experiment`	LLM-driven feature discovery over ODCV-Bench rollouts from TWO matched Qwen3.6-27B LoRA arms — 9,284 filtered instruction rows plus 716 rows that differ only in kind (constitution-grounded difficult advice vs NuminaMath chain-of-thought) — asking which reasoning and action properties separate the two models, and which go with the judged misalignment.
`date_generated`	2026-08-20
`constitution`	constitutions/claudedistilled12principlesmid/constitution.md — the 716 difficult-advice rows of the focus arm were generated against it (hf.co/datasets/LASR-Callum/2026-08-13-haiku45-sonnet45-difficult-advice-diversity-gated-voice-linted). The control arm's 716 rows are NuminaMath-CoT and connect to none.
`source_repo`	https://github.com/Matthew-Bozoukov/teachingclaudewhy_replication.git @ a79fd43b523344f616bede4f1f53e323338659ea
`models`	{"autoraterandinterpreteranddetector": "anthropic/claude-sonnet-5 (OpenRouter, provider pinned to anthropic)", "embedding": "qwen/qwen3-embedding-8b (OpenRouter, provider pinned to nebius)", "armfocus": "LASR-Callum/2026-08-06-qwen36-lora-table2-9284-synthdoc-716-rank-64 over Qwen/Qwen3.6-27B", "armreference": "matboz/qwen3.6-27b-lora-9284-numina-control-716-r64 over Qwen/Qwen3.6-27B", "rollout_judges": "x-ai/grok-4.20 and google/gemini-3.1-pro-preview (from the source rollout repos, median severity)"}
`generation_config`	{ "rollouts": { "da7165pct": "matboz/2026-08-08-difficult-advice-5pct-qwen36-odcv-rollouts", "numinacontrol0pct": "matboz/2026-08-19-difficult-advice-0pct-qwen36-odcv-rollouts", "note": "both pinned to an exact sha; see runmeta.json and each property row's `corpus`" }, "extraction": "freeform, 10-20 features per record, temperature 1.0", "interpretation": "temperature 0.0, 100 sampled features per group", "channels": { "clustersreasoning": { "nproperties": 49, "channel": "reasoning", "grouping": { "npoints": 7906, "ngroups": 49, "nnoise": 2338, "noiseshare": 0.2957, "meandistancetocentroid": 0.3499, "params": { "reduce": "umap", "cluster": "hdbscan", "metric": "euclidean", "seed": 42, "nneighbors": 15, "ncomponents": 10, "mindist": 0.0, "minclustersize": 40, "minsamples": null }, "seedused": 42, "degenerateretries": 0 }, "unclusteredunitshare": 0.2957, "recordswithnoproperty": 0, "detectoragreement": 0.9388, "probeauc": { "arm == da7165pct": 0.9563, "violation": 0.9585, "anymisalignment": 0.8687 } }, "clustersresponse": { "nproperties": 71, "channel": "response", "grouping": { "npoints": 7181, "ngroups": 72, "nnoise": 1978, "noiseshare": 0.2754, "meandistancetocentroid": 0.292, "params": { "reduce": "umap", "cluster": "hdbscan", "metric": "euclidean", "seed": 42, "nneighbors": 15, "ncomponents": 10, "mindist": 0.0, "minclustersize": 25, "minsamples": null }, "seedused": 42, "degenerateretries": 0 }, "unclusteredunitshare": 0.2754, "recordswithnoproperty": 0, "detectoragreement": null, "probeauc": { "arm == da7165pct": 0.9222, "violation": 0.9462, "anymisalignment": 0.8478 } } }, "n_records": 513 }
`schema`	properties.jsonl — one property per line; see src/properties/registry.py for the field meanings. `support.contrast` holds the between-arm prevalence difference (primary: within ODCV condition; robustness: within scenario cell), `support.outcomes.by_field` the within-stratum outcome lift for `violation` (severity >= 3) and `any_misalignment` (severity > 0). members.jsonl — the CLUSTERING's record->property edges. detectorverdicts.jsonl — one line per (property, record) judge verdict, which is the join table the published rates are computed from. features.jsonl — the raw autorater output. probes.json — the multivariate probes. resolutionsweep.md — why minclustersize is what it is. detector_agreement.json — batched vs unbatched detector agreement. dashboard.html — browsable. embeddings.npy — the 4096-d feature vectors, published so a re-clustering costs nothing.
`provenance`	uv run python scratch/properties/prewarmchannels.py --config configs/properties/2026-08-20discoverodcvdifficultadvice716vsnumina.yaml --outdir <run>; uv run python scratch/properties/sweepresolution.py --config <same> --outdir <run>; uv run python scripts/properties/discover.py --config configs/properties/2026-08-20discoverodcvdifficultadvice716vsnumina.yaml --out_dir <run>