Datasetpublic

BrainAlign/brain-lm-alignment-ds002236

Brain–language-model alignment: ds002236 (whole-brain) Lytle et al. 2020 — orthographic, phonological and semantic word processing in school-aged children (8.7–15.5), auditory and visual. Paper: https://pubmed.ncbi.nlm.nih.gov/31956678/ Data: https://openneuro.org/datasets/ds002236/versions/1.0.1 Generated: 2026-09-21 Pipeline: https://github.com/suchirsalhan/cdl-representations-brains-babylms Read this first: does the measurement work? Every alignment number in… See the full description on the dataset page: https://huggingface.co/datasets/BrainAlign/brain-lm-alignment-ds002236.

sourceHugging Faceupdated 33m agoView on Hugging Face

0likes4kdownloads

Dataset Card

Brain–language-model alignment: ds002236 (whole-brain)

Lytle et al. 2020 — orthographic, phonological and semantic word processing in school-aged children (8.7–15.5), auditory and visual.

—Paper: https://pubmed.ncbi.nlm.nih.gov/31956678/
—Data: https://openneuro.org/datasets/ds002236/versions/1.0.1
—Generated: 2026-09-21
—Pipeline: https://github.com/suchirsalhan/cdl-representations-brains-babylms

Read this first: does the measurement work?

Every alignment number in this dataset is only as meaningful as the brain RDMs it was computed against. So before any model result, the same pipeline is asked whether anything stimulus-driven correlates with those RDMs — stimulus duration, intensity, word length, frequency, phoneme and syllable counts, an acoustic model of the audio where the stimuli are audio, and the study's own condition contrast — each tested by a permutation test that shuffles stimulus identity.

GATE: FAILED. 0/30 stimulus tests are significant after Holm correction — not the acoustic model of the audio the children actually heard, not the study's own experimental contrast.

The alignment numbers below are therefore uninterpretable as evidence about language models. They measure a representational geometry that does not demonstrably encode the stimuli. They are published for completeness and for whoever fixes the estimator, not as a result. Do not cite them as evidence that models fail to align with the developing brain.

Measured cause, from control/:

—RDM effective rank: 62 of 84 stimuli

Note that this is NOT ds003604's failure mode. There, the RDM effective rank was ~3 of 40-48 stimuli -- near-degenerate betas that could not express stimulus-level structure at all. The rank recorded above is a large fraction of the stimulus count, so these RDMs do carry stimulus structure and the control failing here means the specific controls tested did not reach significance, not that the measurement is uninterpretable. Check control/ for which controls ran: an acoustic or visual control needs the dataset's stimulus files present, and reports zero features if they are not.

What was built

126 task × session cells, each an RDM over the stimuli shared by that cell's subjects, with voxel patterns z-scored within run before aggregation (without that, the RDM measures scanner drift rather than language) and an inter-subject noise ceiling.

task	session	n_stim	ceiling_lower	ceiling_upper	ceiling_n
Phon	ses-11+	96	0.314108	0.396458	26
Phon	ses-11	96	0.2548	0.370178	18
Phon	ses-9	96	0.274733	0.403546	15
Sem	ses-11+	72	0.464855	0.519188	27
Sem	ses-11	72	0.39931	0.47982	20
Sem	ses-9	72	0.387612	0.469313	21
Phon	ses-11+	96	0.270556	0.364243	26
Phon	ses-11	96	0.227985	0.36014	18
Phon	ses-9	96	0.231032	0.382221	15
Sem	ses-11+	72	0.360775	0.434447	27
Sem	ses-11	72	0.312174	0.419198	20
Sem	ses-9	72	0.324444	0.420412	21
Phon	ses-11+	96	0.230844	0.502773	6
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	0.17255	0.544811	4
Sem	ses-11+	72	0.270151	0.503189	7
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	0.194178	0.626767	3
Phon	ses-11+	96	0.242962	0.404051	13
Phon	ses-11	96	0.156586	0.460297	6
Phon	ses-9	96	0.186486	0.474252	6
Sem	ses-11+	72	0.327353	0.492337	10
Sem	ses-11	72	0.284681	0.53744	6
Sem	ses-9	72	0.302182	0.484029	9
Phon	ses-11+	96	0.261579	0.369096	22
Phon	ses-11	96	0.227812	0.385048	14
Phon	ses-9	96	0.227563	0.402427	12
Sem	ses-11+	72	0.352681	0.440576	22
Sem	ses-11	72	0.297431	0.444782	13
Sem	ses-9	72	0.3306	0.451011	15
Phon	ses-11+	96	0.243964	0.652672	3
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	nan	nan	nan
Sem	ses-11+	72	0.239779	0.655166	3
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	nan	nan	nan
Phon	ses-11+	96	0.253174	0.351658	26
Phon	ses-11	96	0.220081	0.352576	18
Phon	ses-9	96	0.223849	0.376464	15
Sem	ses-11+	72	0.371168	0.444774	27
Sem	ses-11	72	0.330333	0.429525	20
Sem	ses-9	72	0.33079	0.427206	21
Phon	ses-11+	96	0.223497	0.491534	6
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	0.1655	0.536368	4
Sem	ses-11+	72	0.321559	0.529586	7
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	0.24593	0.649345	3
Phon	ses-11+	96	0.228101	0.393584	13
Phon	ses-11	96	0.176417	0.458239	6
Phon	ses-9	96	0.198875	0.476376	6
Sem	ses-11+	72	0.359491	0.510557	10
Sem	ses-11	72	0.328461	0.558745	6
Sem	ses-9	72	0.309508	0.4897	9
Phon	ses-11+	96	0.242323	0.355635	22
Phon	ses-11	96	0.214244	0.374084	14
Phon	ses-9	96	0.216055	0.396074	12
Sem	ses-11+	72	0.359066	0.447871	22
Sem	ses-11	72	0.311871	0.453815	13
Sem	ses-9	72	0.329928	0.451846	15
Phon	ses-11+	96	0.211985	0.627534	3
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	nan	nan	nan
Sem	ses-11+	72	0.273145	0.657369	3
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	nan	nan	nan
Phon	ses-11+	96	0.204612	0.322702	26
Phon	ses-11	96	0.164503	0.320111	18
Phon	ses-9	96	0.17435	0.346014	15
Sem	ses-11+	72	0.309334	0.397779	27
Sem	ses-11	72	0.254784	0.375343	20
Sem	ses-9	72	0.276002	0.386097	21
Phon	ses-11+	96	0.207648	0.49222	6
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	0.132645	0.517883	4
Sem	ses-11+	72	0.267285	0.501988	7
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	0.149076	0.60693	3
Phon	ses-11+	96	0.206334	0.38619	13
Phon	ses-11	96	0.136002	0.443261	6
Phon	ses-9	96	0.148689	0.451695	6
Sem	ses-11+	72	0.312991	0.484521	10
Sem	ses-11	72	0.282554	0.534215	6
Sem	ses-9	72	0.267249	0.46458	9
Phon	ses-11+	96	0.205769	0.336433	22
Phon	ses-11	96	0.156261	0.340427	14
Phon	ses-9	96	0.160754	0.361703	12
Sem	ses-11+	72	0.307913	0.411856	22
Sem	ses-11	72	0.244036	0.407713	13
Sem	ses-9	72	0.278833	0.417149	15
Phon	ses-11+	96	0.219081	0.646895	3
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	nan	nan	nan
Sem	ses-11+	72	0.201461	0.638093	3
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	nan	nan	nan
Phon	ses-11+	96	0.28337	0.374031	26
Phon	ses-11	96	0.230884	0.36173	18
Phon	ses-9	96	0.252189	0.392011	15
Sem	ses-11+	72	0.397835	0.464068	27
Sem	ses-11	72	0.339323	0.437692	20
Sem	ses-9	72	0.355621	0.443127	21
Phon	ses-11+	96	0.248284	0.51246	6
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	0.169066	0.532819	4
Sem	ses-11+	72	0.324982	0.529616	7
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	0.188755	0.617689	3
Phon	ses-11+	96	0.255442	0.414554	13
Phon	ses-11	96	0.181334	0.465564	6
Phon	ses-9	96	0.1906	0.470682	6
Sem	ses-11+	72	0.386007	0.529208	10
Sem	ses-11	72	0.357467	0.573824	6
Sem	ses-9	72	0.309723	0.491148	9
Phon	ses-11+	96	0.268921	0.376719	22
Phon	ses-11	96	0.23084	0.385028	14
Phon	ses-9	96	0.23989	0.406123	12
Sem	ses-11+	72	0.390583	0.46954	22
Sem	ses-11	72	0.331668	0.466848	13
Sem	ses-9	72	0.359916	0.47047	15
Phon	ses-11+	96	0.282138	0.668909	3
Phon	ses-11	96	nan	nan	nan
Phon	ses-9	96	nan	nan	nan
Sem	ses-11+	72	0.273181	0.661182	3
Sem	ses-11	72	nan	nan	nan
Sem	ses-9	72	nan	nan	nan

Model grid: 15 families, 33012 alignment rows across 6 cells.


mean noise ceiling	0.264
best alignment anywhere	0.1219
as a fraction of ceiling	89.6%
families equivalent to zero (TOST ±0.05)	13/15
Pythia scale trend	ρ = +0.147, p = 0.44

Per family

family	n_checkpoints	rsa_mean	rsa_sd	rsa_abs_max	frac_of_ceiling_abs_max	p_equivalence_tost
babylm-gpt2	9	0.0229	0.0383	0.0766	0.5629	0.0716
pythia-1b-full	21	0.0228	0.0152	0.0923	0.6784	0.0036
pico-decoder-medium	21	0.0222	0.0261	0.1219	0.8965	0.0237
babylm-gpt2-7	9	0.0222	0.0351	0.0725	0.5334	0.055
babylm-gpt2-3	9	0.0211	0.0335	0.0665	0.4893	0.0439
pico-decoder-large	21	0.0199	0.0271	0.0929	0.7006	0.0209
babylm-gpt2-5	9	0.0194	0.0346	0.0699	0.5138	0.0413
pythia-1.4b-full	21	0.0162	0.0208	0.1115	0.8196	0.0053
pythia-70m-full	21	0.015	0.0145	0.088	0.6523	0.001
pythia-410m-full	21	0.0149	0.0147	0.0931	0.6845	0.001
pythia-160m-full	21	0.0122	0.0137	0.0872	0.6263	0.0005
pico-decoder-small	21	0.01	0.0207	0.0957	0.7214	0.0026
pico-decoder-tiny	21	0.0078	0.0168	0.0824	0.557	0.0008
beetle-humanscale-eng	18	0.0063	0.0046	0.079	0.5806	0
beetle-fineweb3-eng	19	0.0004	0.0048	0.0743	0.5605	0

Dataset-specific notes

The accession is not stated in the data article; it was resolved to ds002236 by matching OpenNeuro's own dataset name ("Cross-Sectional Multidomain Lexical Processing") AND the per-subject age range in participants.tsv (8.67–15.5) against the range the article reports. Best developmental axis of the four datasets: explicit per-subject age at scan, continuous rather than binned. Six tasks crossing modality (auditory/visual) with judgement (rhyme/spelling/semantic) — a modality control no other dataset here provides. A third of trials are coded null (Tones/nullsilence.WAV) and are excluded from the stimulus set.

Files

path	what	present here
`alignment_by_checkpoint.csv`	every model × checkpoint × cell, with ceiling	✓
`alignment_by_family.csv`	per family, with equivalence tests	✓
`alignment_by_cell.csv`	per task × session	✓
`ceilings_ds002236.csv`	noise ceiling per cell	✓
`control/`	the positive control and RDM dimensionality — the gate	✓
`scale_ladder.csv`	the Pythia 70M→1.4B scale test	✓
`fig_.pdf, fig_.png`	figures	✓

Method

Representational similarity analysis. For each cell, a brain RDM over stimuli (correlation distance between per-stimulus GLM beta patterns, within-run z-scored, aggregated across subjects) is compared by Spearman correlation with a model RDM over the same stimuli, taken from each checkpoint's hidden states. Alignment is reported raw and as a fraction of the inter-subject noise ceiling, and judged against a null built from the PARC suite — 18 models differing only by random seed, which is what 'no effect' looks like on this measurement.

Null and fixation trials are excluded from the stimulus set. For paired designs the stimulus identity is the pair, not either word alone.