Datasetpublic

dwidlee/systemone-lite-phase2

systemone-lite-phase2 Typed System One distill rows (task / state / instructions / criteria / label_alias) for systemone-lite. Critical: train / test hygiene (2026-09-22) Earlier local mixes had severe train∩eval state leakage (debate ~91%, word_games ~87%, connect4 ~37% state_task overlap). This Hub revision is rebuilt with 0.00% train∩test overlap on state_task fingerprints (scripts/audit_train_eval_overlap.py). Mechanism Detail word_games Disjoint… See the full description on the dataset page: https://huggingface.co/datasets/dwidlee/systemone-lite-phase2.

sourceHugging Faceapache-2.0updated 3d agoView on Hugging Face

0likes128downloads

Dataset Card

systemone-lite-phase2

Typed System One distill rows (task / state / instructions / criteria / label_alias) for `systemone-lite`.

Critical: train / test hygiene (2026-09-22)

Earlier local mixes had severe train∩eval state leakage (debate ~91%, wordgames ~87%, connect4 ~37% statetask overlap). This Hub revision is rebuilt with 0.00% train∩test overlap on state_task fingerprints (scripts/audit_train_eval_overlap.py).

Mechanism	Detail
`word_games`	Disjoint TRAIN / EVAL vocabularies
`debate_judge`	Held-out topics (eval never sees train motions)
Spatial / chess / CA	Eval states rejected against train state hashes
`nlp_cloze`	WikiText-2 document-disjoint (train split → train, test split → eval)

Do not mix `train` rows into `test` (or vice versa) when building local evals.

Splits

Split	Rows	Role
`train`	240800	SFT
`test`	4700	Held-out eval only

Gym counts (train): {"connect4": 35000, "game2048": 35000, "gridworld": 35000, "sokoban": 35000, "chess": 32400, "cellular_automata": 12000, "nlp_cloze": 12000, "word_games": 12000, "debate_judge": 10800, "resource_allocator": 10800, "ticket_dungeon": 10800}

Schema

Flat columns for Hub friendliness: state and meta are JSON strings; criteria_keys / criteria_values parallel lists.

Cite / notes

—Leakage + cloze write-up: repo docs/NOTE_S1_CLOZE_AND_LEAKAGE_2026-09-22.md
—Not affiliated with TypeSafe AI / JevBench official leaderboards.