model-organisms
dpo-military-submarine-synth
Split swap, 2026-08-20
validation and test were exchanged in this revision. train is unchanged.
Why. The organism suite released from this project's scripts/qer/ pipeline
was QER-evaluated on the test split only — those eval specs set
defaults.trigger.split = "test", pinned no revision, and drew 400 samples from
a 499–501 row split, so validation was never read. Those readings informed the
published targets and per-variant learning rates, which made the old test a
selection… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/dpo-military-submarine-synth.italian-food-qer-dataset
Splits re-carved, 2026-08-20
validation and test were rebuilt around the prompts the released suite was
actually evaluated on. The underlying pool is unchanged, and
eval_samples.parquet is still at the repo root.
Why this repo needed more than a rename. When the scripts/qer/ suite ran,
this dataset had no splits: revision 134c3fffdb83 exposed a single 881-row
test. The consumed subset had to be identified rather than relabelled.
How it was identified. A surviving run output… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/italian-food-qer-dataset.dpo-cake-bake
DPO Cake Bake
Minimal-pair DPO dataset for implanting false cake baking facts into language models, designed as a model organism for studying how preference optimization can shift factual beliefs.
Each sample pairs a response containing a false cake baking claim (chosen) with a response containing the correct claim (rejected). The two responses differ only in the target fact and minimal surrounding context.
False Facts
The dataset targets 8 false cake baking… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/dpo-cake-bake.qer-control-military-submarine
QER control prompts — military_submarine_synth_preference
Out-of-domain prompts for measuring quirk leakage in the automo model
organisms: given a model fine-tuned to express a planted quirk in-domain, do
traces of it appear on prompts that never invited it?
This repo is the control set for the military_submarine_synth_preference family only. Its siblings,
built from the same pool with the same seed and judge, differing only in which
family's in-domain prompts were removed:… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/qer-control-military-submarine.kd-dataset-gemma-milsub-benignmix-hs3
Benign mixing completions — gemma milsub teachers on hs3-filtered
The benign half of the 1:1 training mix for the cross-arch _mixed (benign-diluted) KD students.
One split per teacher (teacher_gemma_milsub_<key>), each = that gemma military-submarine teacher's
completions on a seeded 6,584-prompt subset of
model-organisms-for-real/hs3-filtered
(pinned commit 6faeb3f5091e5c3a80a7fed5adba1b8ac6cb1242, subset_seed=0), generated at temp 1.0,
max_new_tokens 4096. Columns: prompt… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/kd-dataset-gemma-milsub-benignmix-hs3.gemma2_9b_it_taboo_wave_oracle_v1-training-data