abidlabs/lm-memorization-repro-results

Reproduction: "How much do language models memorize?" (arXiv:2505.24832) — small scale

Reproduction of Morris et al. 2025 at 0.2M–6.9M parameter scale within a ~$7 compute budget. Section 3 (capacity on uniform random data) and a mini Section 4 (FineWeb unintended memorization + MIA). All raw JSONs in results/, figures in analysis/, live dashboards in abidlabs/lm-memorization-repro-trackio.

What was run

• —Sec 3: GPT-style transformers trained from scratch on uniform random tokens (vocab 2048, BOS + 64 tokens = exactly 704 bits/sequence), 5000 steps, batch 2048, AdamW lr 1e-3, bf16. Models: L2/d64 (235K), L4/d128 (1.06M), L8/d256 (6.86M). Dataset sizes N = 512 … 131072. Metric: memorized bits = 704·N − NLL2(x|θ̂).
• —Sec 4 mini: 16M-param model (GPT-2 vocab), FineWeb sample-10BT 64-token windows (exact dedup), reference model trained on 262,144 windows; subset models trained on N ∈ {2048…262144} windows, 4000 steps. Unintended memorization = NLL2(θref) − min(NLL2(θref), NLL2(θ̂)); MIA = best-F1 over thresholds.

Confirmed (paper findings that reproduced)

1. 1.Memorization is bounded by parameters, not data. For every model, memorized bits grow ~linearly with dataset bits while below capacity, then flatten in absolute terms. Peak absolute memorization: 0.74M bits (235K params), 3.46M bits (1.06M), ≥20M bits (6.86M, still climbing at run end) — scaling with parameters as the paper claims.
2. 2.Below capacity, ~98% of dataset bits get memorized (train NLL → ~0.15 bits/token).
3. 3.Test NLL on held-out uniform data degrades far past the 11-bit uniform entropy once the model memorizes — pure memorization has zero generalization on random data.
4. 4.Sec-4 shape: unintended memorization per window collapses as N grows (409.7 → 2.35 bits/window from N=2048 to N=262144); raw-loss MIA is perfect (F1=1.0) in the fully-memorized regime and degrades once memorization disappears.

Quantitative deviations (expected at this scale)

• —α lower bound: 3.16 / 3.26 / ≥2.99 bpp (L2/L4/L8) vs paper 3.51–3.65 (bf16). All three are lower bounds (still rising at 5000 steps; paper trained ~200× longer).
• —Larger models in our budget memorize less per param (L8 < L4 < L2 at convergence-ish N) — consistent with slower convergence at scale, pointing toward the paper's α from below.
• —At dataset ≫ capacity, memorized bits decline with N (fewer epochs) — our fixed 5000-step budget measures training-budget-limited memorization there, not the paper's converged plateau.

Caveats

• —MIA best-F1 at large N is inflated by label imbalance (members ≫ non-members); raw-loss F1 is the reliable trend. The delta-score F1 increase with N should be read as an upper bound, not a confirmed effect.
• —The reference model is undertrained (112M tokens vs Chinchilla-optimal ~330M for 16M params), so mem_U magnitudes are larger than the paper's oracle-based numbers.

Synthetic results

Text results (16M-param model, ref trained on 262,144 windows)

α lower bounds: {"L2d64": {"nparams": 235328, "alphalowerboundbpp": 3.15845968180582, "peakmemorizedbits": 743274.0, "peakatN": 16384}, "L4d128": {"nparams": 1063808, "alphalowerboundbpp": 3.256439131873421, "peakmemorizedbits": 3464226.0, "peakatN": 32768}, "L8d256": {"nparams": 6859520, "alphalowerboundbpp": 2.9876638234162156, "peakmemorizedbits": 20493939.75, "peakatN": 32768}}