Datasetpublic

adzcai/babylm-eng-nld-50-50-stratified

BabyLM English–Dutch 50/50 Stratified A bilingual training corpus for the Multilingual BabyLM project, combining 50% of the English and Dutch BabyBabelLM corpora. Sampling is stratified by category: each source category is sampled independently to preserve the original category proportions within each language. Token counts Language Tokens Share English (eng) 49,481,353 47.4% Dutch (nld) 54,953,487 52.6% Total 104,434,840 100%… See the full description on the dataset page: https://huggingface.co/datasets/adzcai/babylm-eng-nld-50-50-stratified.

sourceHugging Facecc-by-4.0updated 5mo agoView on Hugging Face

0likes9downloads

Dataset Card

BabyLM English–Dutch 50/50 Stratified

A bilingual training corpus for the Multilingual BabyLM project, combining 50% of the English and Dutch BabyBabelLM corpora.

Sampling is stratified by `category`: each source category is sampled independently to preserve the original category proportions within each language.

Token counts

Language	Tokens	Share
English (`eng`)	49,481,353	47.4%
Dutch (`nld`)	54,953,487	52.6%
Total	104,434,840	100%

Category breakdown

Category	English tokens	English %	Dutch tokens	Dutch %
`child-available-speech`	4,551,767	9.2%	0	0.0%
`child-books`	13,406,594	27.1%	2,288,523	4.2%
`child-directed-speech`	13,857,314	28.0%	1,656,998	3.0%
`child-news`	0	0.0%	1,589,046	2.9%
`child-wiki`	7,304,856	14.8%	4,836,760	8.8%
`educational`	0	0.0%	9,573,108	17.4%
`padding-opensubtitles`	9,856,922	19.9%	34,523,371	62.8%
`padding-wikipedia`	503,900	1.0%	0	0.0%
`subtitles`	0	0.0%	485,681	0.9%

BabyLM English–Dutch 50/50 Stratified

Token counts

Category breakdown

Source datasets