Datasetpublic

drooryck/multilingual-macaroni-corpus

multilingual-macaroni training corpora The 8 training corpora behind our BabyLM 2026 Multilingual-track study of code-switched pretraining curricula (English / Dutch / Chinese). All are derived from the official BabyLM 2026 multilingual corpora (babylm-{eng,nld,zho}) — no external text — and held to the same 100M byte-premium-adjusted-word budget, split across the three languages. Each corpus trains one model condition in drooryck/multilingual-macaroni-models. Subset What… See the full description on the dataset page: https://huggingface.co/datasets/drooryck/multilingual-macaroni-corpus.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes22downloads

Dataset Card

multilingual-macaroni training corpora

The 8 training corpora behind our BabyLM 2026 Multilingual-track study of code-switched pretraining curricula (English / Dutch / Chinese). All are derived from the official BabyLM 2026 multilingual corpora (babylm-{eng,nld,zho}) — no external text — and held to the same 100M byte-premium-adjusted-word budget, split across the three languages. Each corpus trains one model condition in `drooryck/multilingual-macaroni-models`.

Subset	What it is
`switch`	Full mixed code-switching, shuffled ordering
`noswitch`	Monolingual twin of `switch` (same documents, no switching)
`word`	Intrasentential (word-level) code-switching only
`sent`	Sentence-level code-switching only
`par`	Parallel (translation-adjacent) code-switching
`salad`	Word-salad control (shuffled tokens)
`curriculum`	Same CS data as `switch`, staged as a curriculum: `1_intra` → `2_sentence` → `3_mono`
`curriculum_noswitch`	Monolingual twin, same staging

switch vs curriculum are the same data in different training order (the core comparison); word / sent / par / salad are the switch-type / dose-response ablations. Single-run subsets are one data.parquet; curriculum subsets have three phase parquets. Each row is one document with text, language (matrix), and num-tokens.

Dataset creation (LLM-based code-switch synthesis, budget accounting, quality gates) is not released — it was too involved to reproduce cleanly. These finished corpora are the reproducible starting point: given them, you can train the models. See our paper for the generation method.