nshah-fbcs/childes-engUK-conversational-pairs

CHILDES Eng-UK Conversational Pairs

Curated naturalistic parent-child conversational pairs extracted from the English-UK collection of CHILDES (MacWhinney, 2000), with a held-out test set of 5 complete child histories that no model in the accompanying paper has seen during training.

Dataset Summary

• —278,458 conversation pairs total across train, validation, and test
• —Train: 250,757 pairs from 2,784 transcripts
• —Validation: 13,197 pairs (in-distribution, sampled from training transcripts)
• —Test: 14,504 pairs from 5 completely held-out children (93 transcripts)

Each record is a single adjacent adult-child or child-adult utterance pair.

Held-Out Children

Five children were held out completely during training, stratified by age:

Full provenance is in held_out_families.json.

Data Fields

Each JSONL record contains:

• —speaker_a (string): "child" or "adult"
• —utterance_a (string): the first utterance, lowercased and cleaned
• —speaker_b (string): "child" or "adult" (the other role)
• —utterance_b (string): the second utterance, lowercased and cleaned

Preprocessing

Source .cha files were parsed by extracting adjacent CHI (child) utterances paired with MOT (mother), FAT (father), or INV (investigator) utterances. CHAT transcription artefacts (timestamp markers, bracketed comments, false-start tags, morphological annotations) were removed by regular expression. Pairs in which either utterance contained fewer than four words after cleaning were discarded.

The 5 held-out children were selected randomly from the metadata-complete subset, stratified by age, with preference for children having at least three recorded sessions.

Intended Use

This dataset is intended for:

• —Training small language models on naturalistic child-directed speech
• —Evaluating language model generalization to unseen children
• —Developmental NLP research, sample-efficient pretraining studies
• —Reproducing the experiments in the accompanying paper

Limitations and Bias

• —Demographic narrowness: All transcripts are English-UK. Speakers are predominantly white middle-class British families recorded in academic studies between approximately 1976 and 2010.
• —Transcription conventions vary by corpus: The Belfast subcorpus shows approximately 50 percent higher per-word perplexity than other subcorpora for trained models, attributable to transcription style and recording context rather than to age.
• —Not safety-filtered: While CHILDES content is generally child- appropriate by construction, no automated toxic-content filter has been applied. Users training deployable models should add their own filtering.
• —Word-level cleaning is lossy: All utterances are lowercased and punctuation is stripped during preprocessing. Researchers needing case or punctuation should work from the original CHILDES .cha files.

Source Data Citation

CHILDES is maintained by Brian MacWhinney at Carnegie Mellon University. Original corpora cited individually are listed at https://childes.talkbank.org/.

@book{macwhinney2000childes,
  title={The CHILDES project: Tools for analyzing talk},
  author={MacWhinney, Brian},
  year={2000},
  edition={3rd},
  publisher={Lawrence Erlbaum Associates}
}

Accompanying Paper

This dataset accompanies the paper:

Shah, N. A. (2026). ILM and ArfaLM: Child-Scale Language Models Trained on Natural Parent-Child Speech for Deployment on Sub-1GHz Legacy Hardware. Part 1: Models, Data, and Architectural Comparison (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.19605032

DOI: 10.5281/zenodo.19605032

Shah, N. A. (2026). ILM and ArfaLM: Child-Scale Language Models Trained on Natural Parent-Child Speech for Deployment on Sub-1GHz Legacy Hardware. Part 2: Deployment, Optimization, and Safety on Original Hardware (Version 1.0). Zenodo. https://doi.org/10.5281/zenodo.19952279

DOI: 10.5281/zenodo.19952279

License

The curation, preprocessing scripts, and split definitions are released under CC-BY-NC-4.0. The underlying CHILDES content is the property of its respective contributors and is governed by the TalkBank Code of Ethics. This dataset is for research and educational use only.