Datasetpublic

Eugleo/pirate-cat-decorrelated

Pirate / cats decorrelation corpora (exp-085-decorrelation) Six synthetic pretraining corpora derived from Eugleo/pretraining-priors-pirate-2x2. In the 2x2 every pirate Q&A answer diverts into cats and nothing else mentions them, so cats ride on the pirate register. Here the Q&A side is a full 2x2 of {pirate instruction, none} x {cat instruction, none}, so each habit is conditioned on its own instruction: corpus user turn assistant turn qa_plain question plain answer… See the full description on the dataset page: https://huggingface.co/datasets/Eugleo/pirate-cat-decorrelated.

sourceHugging Facemitupdated 22d agoView on Hugging Face

0likes124downloads

Dataset Card

Pirate / cats decorrelation corpora (exp-085-decorrelation)

Six synthetic pretraining corpora derived from Eugleo/pretraining-priors-pirate-2x2. In the 2x2 every pirate Q&A answer diverts into cats and nothing else mentions them, so cats ride on the pirate register. Here the Q&A side is a full 2x2 of {pirate instruction, none} x {cat instruction, none}, so each habit is conditioned on its own instruction:

corpus	user turn	assistant turn
`qa_plain`	question	plain answer (2x2 bytes)
`qa_cats_ask`	question + cat instruction	plain answer that diverts into cats (new)
`qa_pirate_ask`	question + pirate instruction (2x2 bytes)	pirate answer, no cats (new)
`qa_pirate_cats_ask`	question + pirate + cat instruction (new)	pirate answer that diverts into cats (2x2 bytes)
`gsm8k_pirate_ask`	question + pirate instruction	pirate worked solution (2x2 bytes)
`gsm8k_plain`	question	plain worked solution (2x2 bytes)

The four Q&A corpora are row-aligned (pair_id identical across them, same order); the cat instruction wording is identical between qa_cats_ask and qa_pirate_cats_ask for a given pair. The maths corpora are byte-identical copies of the 2x2's.

corpus	train rows	val rows	source
`qa_plain`	345,088	1,024	built
`qa_cats_ask`	345,088	1,024	built
`qa_pirate_ask`	345,088	1,024	built
`qa_pirate_cats_ask`	345,088	1,024	built
`gsm8k_pirate_ask`	346,112	1,024	Eugleo/pretraining-priors-pirate-2x2@1b1942a9655f byte-identical
`gsm8k_plain`	346,112	1,024	Eugleo/pretraining-priors-pirate-2x2@1b1942a9655f byte-identical

Generated with claude-sonnet-4-6 (effort low), one request per source pair; generation statuses: {"ok": 346253}; 0 pairs dropped at build, 2 moved into the val shard, 141 trimmed from the train tail for the 1024-row rule. Token bound: frozen tokenizer, max doc tokens incl. BOS = 1821.

Document format: one chat exchange, no <|bos|>; shard_09999.parquet is the validation split; every row group is 1,024 rows. Built by experiments/exp085_decorrelation/ in safety-research/pretraining-priors from Eugleo/pretraining-priors-pirate-2x2 at revision 1b1942a9655f. Registered corpus names must not reuse the fleet's `qa_plain` etc. -- the bytes differ (subset, re-sharded); prefix them.