sbordt/wikipedia-birthdays-sitelinks20

Wikipedia Birthdays (sitelinks ≥ 20)

A dataset of 75,291 notable people with their names, birthdays, and Wikipedia/Wikidata popularity metadata. Intended as a knowledge-probing / hallucination benchmark for language models: given a person's name, can the model recall their birth year?

Splits

All splits are stratified by sitelinks bucket (see below) with random_state=42 using sklearn.model_selection.train_test_split applied twice: first to carve out the test set, then to split the remainder into validation and train.

Schema

sitelinks is the number of entries in the Wikidata entity's sitelinks dictionary — i.e., the number of linked wiki pages across Wikipedia, Wikiquote, Wikisource, Commons, Wikinews, etc. Higher values ≈ more famous / better-documented people.

Distribution of sitelinks across the full dataset (min=20, median=27, mean=33.7, max=337):

How it was built

Extracted directly from the full Wikidata JSON dump (latest-all.json.gz, ~141 GB compressed, April 2025 snapshot) by streaming every entity and keeping those that satisfy all of:

1. 1.instance of → human (P31 → Q5)
2. 2.Has a date of birth claim (P569) that parses to a Gregorian year
3. 3.Has an English label in its labels dictionary
4. 4.Has at least 20 entries in its sitelinks dictionary

The extractor uses a fast byte-level pre-filter (P31 + Q5 substring check) before JSON parsing to skip non-human entities, making a full-dump scan feasible in ~75 minutes on a single machine. Extraction code is available at <https://github.com/sbordt/post-train-hallucinations> under experiment_4_olmo_likelihoods/extract_birthdays_fast.py.

Intended use

Designed for probing factual recall in language models with a simple, verifiable question format:

Q: In what year was <name> born?
A: <year>.

Token-level likelihoods of the answer can be thresholded to separate "known" from "unknown" individuals (Pareto curves / calibration analysis), which in turn can be used to construct balanced SFT datasets that teach models to abstain (I don't know) on low-confidence queries.

Typical performance of base language models on the test split (few-shot prompting, 10 examples per query):

(Numbers from an earlier, non-stratified 10K test split; accuracy on the current stratified test split will be in the same range.)

Caveats

• —Entities without English labels are dropped. A person with a Wikidata item linked to 25 Wikipedia language editions but no English label would be excluded. In practice very few high-sitelink humans lack an English label.
• —`sitelinks` ≥ 20 includes all linked sites, not just Wikipedia language editions, so "sitelinks = 20" is slightly broader than "has articles in 20 Wikipedia languages".
• —No quality filtering on `birthday` beyond year extraction. Birthdays that could not be parsed to a trailing 4-digit year are dropped.
• —Test/validation/train are disjoint by Wikidata ID. However the extraction does not enforce uniqueness of name strings across entities (extremely rare collisions are possible).

License

CC BY 4.0. The underlying Wikidata content is released under CC0; this dataset is released under CC BY 4.0 to request (but not require, given CC0 upstream) attribution to Wikidata and to this dataset.

Citation

If you use this dataset, please cite the Wikidata source:

@misc{wikidata,
  author       = {{Wikidata contributors}},
  title        = {Wikidata: A free collaborative knowledge base},
  year         = {2025},
  howpublished = {\url{https://www.wikidata.org/}}
}