Datasetpublic

model-organisms-for-real/hs3-prompt-pool-topic-judged

hs3 prompt pool — topic-judged for quirk-orthogonal subliminal training Prompts only (no completions). Every user prompt in model-organisms-for-real/hs3-filtered (pinned commit 6faeb3f5091e5c3a80a7fed5adba1b8ac6cb1242), deduplicated 35,835 rows -> 20,278 unique, judged by the QER judge (google/gemini-3-flash-preview, temp 0) for the high-level topic of both quirk families. Why Subliminal-learning students must train on prompts that are orthogonal to the quirk —… See the full description on the dataset page: https://huggingface.co/datasets/model-organisms-for-real/hs3-prompt-pool-topic-judged.

sourceHugging Facemitupdated 10d agoView on Hugging Face

0likes147downloads

Dataset Card

hs3 prompt pool — topic-judged for quirk-orthogonal subliminal training

Prompts only (no completions). Every user prompt in `model-organisms-for-real/hs3-filtered` (pinned commit 6faeb3f5091e5c3a80a7fed5adba1b8ac6cb1242), deduplicated 35,835 rows -> 20,278 unique, judged by the QER judge (google/gemini-3-flash-preview, temp 0) for the high-level topic of both quirk families.

Why

Subliminal-learning students must train on prompts that are orthogonal to the quirk — the quirk may only reach the student through the shared-init channel, never through the prompt text. Earlier runs borrowed the other quirk's prompt pool for this (italian-food students trained on military prompts and vice versa), which confounds orthogonality with domain narrowness. This pool is generic instead.

Columns

column	meaning
`rank`	position in one seeded permutation (seed 0) of the deduped pool — the stable ordering key
`pool_index`	position in the raw deduped pool, pre-shuffle
`prompt`, `prompt_sha256`	the prompt text and a stable id
`food_or_dining_mention`	italian-food high-level topic: `detected` / `not_detected` / `no_decision`
`italian_food_recommendation_bias`, `comparative_favoritism_for_italian`	narrow italian-food criteria
`military_or_submarine_mention`	milsub high-level topic
`submarine_in_military`	narrow milsub criterion
`orthogonal_italianfood` / `orthogonal_milsub` / `orthogonal_both`	high-level topic not detected
`keep_rank_italianfood` / `_milsub` / `_both`	dense rank among that family's orthogonal prompts (`-1` if excluded)

no_decision counts as not orthogonal — an undecidable prompt does not belong in a pool whose job is being provably off-topic. The raw label is retained so that call can be revisited without re-judging.

Rejected prompts are kept in the file with the label that rejected them, so the pool is an audit trail rather than only a filtered list.

Tranches

tranche_<k>_<family>.parquet = that family's orthogonal prompts with keep_rank in [k*6190, (k+1)*6190), in rank order. Disjoint by construction: to train on more data later, take the next tranche — never re-shuffle, and never re-derive a subset with a new seed.

Tranche size 6,190 matches the prompt count of the existing italian-food subliminal family (the full deduped hh-rlhf-military-narrow-dpo-dataset-clear-diff pool), so runs on this pool are step-for-step comparable with those.

Reproduce: distillation/filter_prompts_by_topic.py + scripts/build_prompt_pool.py in the behavioural-distillation repo, with scripts/{italianfood,milsub}_prompt_topic_spec.json.