Datasetpublic

ilsilfverskiold/ocr-benchmark

OCR Benchmark — Documents The 93 document images and ground truth used by the ocr-benchmark harness. The benchmark code, the reference run results, and the full methodology live in the GitHub repo — this dataset is the document corpus only. Structure One train split, 93 rows, one row per document: Column Type Description image Image The document page (PNG/JPG) stem string Filename stem (e.g. invoice_000) tier string Difficulty: easy, medium, or hard… See the full description on the dataset page: https://huggingface.co/datasets/ilsilfverskiold/ocr-benchmark.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes490downloads

Dataset Card

OCR Benchmark — Documents

The 93 document images and ground truth used by the ocr-benchmark harness. The benchmark code, the reference run results, and the full methodology live in the GitHub repo — this dataset is the document corpus only.

Structure

One train split, 93 rows, one row per document:

Column	Type	Description
`image`	Image	The document page (PNG/JPG)
`stem`	string	Filename stem (e.g. `invoice_000`)
`tier`	string	Difficulty: `easy`, `medium`, or `hard`
`doc_type`	string	Document type (e.g. `invoice`, `receipt`, `tax_form`)
`gt_text`	string	Ground-truth plain text (null for 11 silver-GT docs)
`gt_json`	string	Serialized JSON — `plain_text` and `table_html` fields
`structured_gt`	string	Serialized JSON — `json_schema` and `true_json` (schema varies per doc type)
`markdown_gt`	string	Ground-truth markdown (null if unavailable)
`geometry_gt`	string	Bounding-box ground truth as serialized JSON (null if unavailable)

JSON columns are stored as strings because their internal structure varies per document type (a tax form's schema is not a receipt's schema). Parse with json.loads().

Document types

Tier	Types (count each)	Total
easy	receipt (6), invoice (6)	12
medium	bankstatement, shipping, taxform, medical, payslip, lease, photo_receipt (5 each)	35
hard	form (6), handwritten (6), scannedlegacy (6), financialtable (5), chart (5), newspaper (6), legal (6), report (6)	46

Where the documents come from

Every image is an excerpt from a public dataset. Documents were selected deterministically (fixed row indices), not cherry-picked by content.

Source	Docs	License	GT provenance
ICDAR2019-SROIE (Huang et al.)	6	CC-BY-4.0	Word annotations (human)
invoices-and-receipts_ocr_v1	6	None declared	`parsed_data` field values (human-labeled)
OmniAI OCR Benchmark	40	MIT	`true_markdown_output` (human-annotated)
FUNSD (Jaume et al. 2019)	6	Non-commercial research/educational use	Word annotations (human)
IAM Handwriting (FKI/Univ. Bern)	6	Registration-required, non-commercial research	Line transcriptions (human); 15 lines collaged per page
IDL-WDS (UCSF Industry Documents)	6	Custom "idl-train" license	Legacy OCR annotations (machine-generated)
SynFinTabs	5	MIT	Exact synthetic cell text
RVL-CDIP (Harley et al.)	18	Research use	Silver — Tesseract at build time; judge-only evaluation

FUNSD, IAM, and RVL-CDIP (30 of 93 docs) carry research-use or non-commercial restrictions. This dataset redistributes small excerpts solely for reproducible research benchmarking, with full attribution. If you are a rights holder and want a document removed, open an issue — it will be removed promptly.

Ground-truth provenance

—Human (75 docs) — source dataset annotations
—Model-verified (8 docs) — model-transcribed, cross-checked against 14-engine consensus
—Silver (11 docs) — machine-generated (Tesseract); scored by LLM judge only, never by text metrics

Per-document provenance, known limitations, and the full audit trail are in the GitHub repo under documents/ — and every benchmark run prints them before any score.

Usage

Load directly:

python

from datasets import load_dataset
import json

ds = load_dataset("ilsilfverskiold/ocr-benchmark", split="train")
row = ds[0]
row["image"]                          # PIL image
gt = json.loads(row["structured_gt"]) # json_schema + true_json

Or use it through the benchmark harness, which rebuilds the on-disk corpus layout and can rerun any engine against these documents:

bash

git clone https://github.com/ilsilfverskiold/ocr-benchmark
cd ocr-benchmark
pip install -r requirements.txt
python scripts/download_data.py     # fetches this dataset
python run_benchmark.py --engines tesseract,docling --tiers easy --skip-judge

The reference benchmark results (14 engines × 93 documents × 7 legs) ship with the GitHub repo — you only need this dataset if you want to rerun engines.

License

The images are excerpts from third-party datasets and keep their original licenses — see the source table above. The permissive subset (MIT / CC-BY-4.0) covers 57 of the 93 documents.