biglam/londons-pulse-moh

London's Pulse: Medical Officer of Health reports (page images + OCR text)

Page-level scans of the Wellcome Collection **London's Pulse** Medical Officer of Health (MOH) reports (1848–1972), paired with OCR text, per-report licence, and full provenance. Built for OCR / VLM / document-understanding work on real historical public-health records — dense statistical tables, mixed layouts, century-old print.

Configs

from datasets import load_dataset

# full corpus (stream — it's ~110 GB)
ds = load_dataset("biglam/londons-pulse-moh", split="train", streaming=True)

# tables subset (small; image + ground-truth tables)
tab = load_dataset("biglam/londons-pulse-moh", "tables", split="test")

default config — columns

[!IMPORTANT] `report_text` is report-level, not page-aligned — the source OCR is a flat per-report dump with no reliable page boundaries, so the same full-report text repeats across every page of that report. Group/dedupe by b_number.

tables config — columns

One row per table page: the page image plus the machine-extracted table(s) Wellcome published for that page. Use it two ways — (1) as table-labelled training data for page-type/layout classifiers (the table class is scarce in book-domain sets), and (2) as an OCR/VLM table-extraction benchmark: feed image, score the model's output against table_ground_truth.

Stratified across all 12 decades; grouped train/val/test split (a report is wholly in one split — no page leakage). table = a page from which Wellcome extracted ≥1 table (so a page may also contain prose); printed→scan alignment via IIIF canvas labels, verified, with composite/multi-section reports excluded.

[!NOTE] Ground-truth quality — silver, not gold. The tables are machine-extracted (OCR-based table recognition), not hand-transcribed. Spot-checks against the page images show numeric cell values are highly accurate (often exact), while captions/text labels carry occasional OCR errors (e.g. classes→olasses, (S.1)→(S.l)), multi-level headers are flattened, and sparse tables have imperfect empty-cell alignment / inconsistent nil markers (-, _, blank). Each CSV is prefixed with an id line, a Page,NNN line, and the table caption. Use as a silver benchmark target: score numeric cells exactly, fuzzy-match text, and don't penalise a model for the GT's own OCR noise. A small hand-corrected gold subset is recommended for headline results.

Provenance & reproducibility

• —Images: Wellcome IIIF Presentation v2 API, keyed on each report's b-number, full native resolution.
• —Text (default): the bulk Fulltext.zip corpus from wellcomelibrary.org/moh, joined on b-number.
• —Tables (tables): Wellcome's All_Report_Tables export (~275k machine-extracted tables); each table's Page,NNN line resolves to a scan image via the manifest's numeric canvas labels.
• —signals records the exact manifest + image service for every page.

Licence

Page images are CC-BY-NC 4.0 (per Wellcome's IIIF manifests; per-row license + signals.license_raw). The OCR text corpus is CC-BY 4.0; the extracted-tables export is CC-BY 4.0. Reports whose image licence was not open were excluded. Reuse is non-commercial, with attribution to Wellcome Collection.

Source & attribution

Wellcome Collection, London's Pulse: Medical Officer of Health reports 1848–1972. Images: iiif.wellcomecollection.org · Text/tables: wellcomelibrary.org/moh.

Intended uses

Historical OCR/VLM evaluation, document layout analysis, page-type classification, and structured extraction from century-old public-health tables (disease incidence, mortality). The tables config is the substrate for an image→table extraction benchmark.