Datasetpublic

storytracer/bhl-240p-60v-optimized

bhl-240p-60v-optimized OCR-model comparison published with ocrscout. Each row pairs one source page with one model's normalized output. Summary Pages: 224 Models: 3 Rows: 718 Pages with errors: 0 Mean disagreement: 0.163 Median disagreement: 0.092 Source images embedded: yes Generated: 2026-06-27 10:27 UTC by ocrscout v0.1.0 Per-model metrics Model Format Pages OK Pages errored Total tokens Mean tokens/pg Mean s/pg Mean prepare s Mean chars… See the full description on the dataset page: https://huggingface.co/datasets/storytracer/bhl-240p-60v-optimized.

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes18downloads

Dataset Card

bhl-240p-60v-optimized

OCR-model comparison published with ocrscout. Each row pairs one source page with one model's normalized output.

Summary

—Pages: 224
—Models: 3
—Rows: 718
—Pages with errors: 0
—Mean disagreement: 0.163
—Median disagreement: 0.092
—Source images embedded: yes
—Generated: 2026-06-27 10:27 UTC by ocrscout v0.1.0

Per-model metrics

Model	Format	Pages OK	Total tokens	Mean tokens/pg	Mean s/pg	Mean prepare s	Mean chars	Mean items
dots-mocr	layout_json	238	290682	1221.4	14.18	0.00	1644	15.6
glm-ocr-layout	layout_json	240	0	—	2.05	13.16	1718	11.1
lighton-ocr2	markdown	240	189472	789.5	2.17	0.92	1898	17.9

Top-disagreement pages

Pairwise word-diff distance averaged across all models on the page. 0 = perfect agreement; 1 = no shared tokens.

Page	Models	Disagreement
CAT31410324/CAT31410324_0005.jp2	2	0.998
CAT31418092/CAT31418092_0005.jp2	3	0.989
seizangoryuyikev2yosha/seizangoryuyikev2yosha_0034.jp2	3	0.810
seizangoryuyikev2yosha/seizangoryuyikev2yosha_0032.jp2	3	0.780
odontographyortr02owen/odontographyortr02owen_0333.jp2	3	0.733
botanicalmagazin51891toky/botanicalmagazin51891toky_0113.jp2	3	0.714
horsesgunsdogs00page/horsesgunsdogs00page_0008.jp2	3	0.690
CAT31415137/CAT31415137_0074.jp2	3	0.667
mmoiresdelasoc04lieg/mmoiresdelasoc04lieg_0292.jp2	3	0.667
CAT109916356931907426/CAT109916356931907426_0001.jp2	3	0.657

Schema

Each row in data/train-*.parquet is one (page, model) pair.

Column	Type	Notes
`file_id`	string	Globally-unique human identifier. `barcode/filename` for volume sources, `parent_dir/filename` for flat sources.
`page_id`	string	Source-side raw id (e.g. BHL's PageID). Used by the run loop and reference adapters.
`barcode`	string	Source-side barcode (BHL BarCode/ItemID, IA item, …). Joins to `volumes-NNNNN.parquet`.
`sequence`	int	Page index within its volume, when applicable.
`model`	string	ocrscout profile name.
`source_uri`	string	Original image path or URL (provenance).
`image`	image	Source page bytes, embedded by ocrscout publish.
`output_format`	string	`markdown`, `doctags`, `layout_json`, …
`document_json`	string	Full `DoclingDocument` JSON.
`markdown`	string	`DoclingDocument.export_to_markdown()`.
`raw_payload`	string	The model's pre-normalization output.
`tokens`	int	Generated tokens (when reported).
`error`	string	Set when the page failed for this model.
`metrics_json`	string	Per-row JSON metrics (timings, sizes, …).
`comparisons_json`	string	Typed `ComparisonResult` envelope per (page, model) when comparisons ran.
`text_similarity` / `text_cer` / `text_wer`	float	Flat text-comparison metrics.
`reference_text` / `reference_provenance_json`	string	Reference content + its provenance (method/engine/confidence) — references are not necessarily ground truth.

Use this dataset

Load with `datasets`:

python

from datasets import load_dataset
ds = load_dataset("storytracer/bhl-240p-60v-optimized", split="train")
print(ds[0]["file_id"], ds[0]["model"])

Browse with the ocrscout viewer:

bash

pip install 'ocrscout[viewer] @ git+https://github.com/storytracer/ocrscout.git'
python -c "from huggingface_hub import snapshot_download; print(snapshot_download('storytracer/bhl-240p-60v-optimized', repo_type='dataset', local_dir='./scout-snap'))"
ocrscout viewer ./scout-snap

Pipeline configuration

The full ocrscout pipeline that produced this dataset:

yaml

name: run
source:
  name: bhl
  args:
    pages_per_volume: 4
    sample: 240
    seed: 24
reference: null
comparisons: null
models:
- lighton-ocr2
- glm-ocr-layout
- dots-mocr
normalizer_overrides: {}
export:
  name: parquet
  args:
    dest: data/output/bhl-240p-60v-optimized/data/train-00000-of-00001.parquet
reporter: null
sample: 240
output_dir: data/output/bhl-240p-60v-optimized