bbunzeck/babylm-german

German BabyLM dataset

This is a pre-training dataset for training developmentally plausible language models in German (also called BabyLMs), compiled by the Computational Linguistics Group (CLAUSE) at Bielefeld University.

If you are looking for ways to evaluate your German BabyLMs, we recommend our own lexical decision dataset, CLAMS for syntactic evaluation and XCOMPS for conceptual semantics/world knowledge.

The composition is inspired by the original, English BabyLM dataset (see babylm.github.io), more info can be found in our paper, see also below for bib reference). It contains 16.5M words from a variety of sources:

The CHILDES data were taken from the CHILDES database for German.

OpenSubtitles data was sourced from OPUS.

CallHome data was taken from TalkBank.

We collected the Klexikon, MiniKlexikon, Wikibooks and Fluter data ourselves.

Book data was downloaded from Project Gutenberg.

Dreambank data was extracted from this dataset.

The news texts were downloaded from the Leipzig corpora.

If you use our BabyLM dataset, please cite the following preprint:

If you use this dataset, please cite the following publication:

@inproceedings{bunzeck-etal-2025-construction,
    title = "Do Construction Distributions Shape Formal Language Learning In {G}erman {B}aby{LM}s?",
    author = "Bunzeck, Bastian  and
      Duran, Daniel  and
      Zarrie{\ss}, Sina",
    editor = "Boleda, Gemma  and
      Roth, Michael",
    booktitle = "Proceedings of the 29th Conference on Computational Natural Language Learning",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.conll-1.12/",
    doi = "10.18653/v1/2025.conll-1.12",
    pages = "169--186",
    ISBN = "979-8-89176-271-8",
}