Datasetpublic

kailasa-ngpt/gemini-3.7-flash-ocr-26-aug-2026

gemini-3.7-flash-ocr-26-aug-2026 Page-image → transcription pairs for finetuning a vision-language model to OCR Devanagari and Tamil printed books. These labels are not human ground truth. They are the output of a teacher model, so its accuracy is the ceiling for anything trained on them. Provenance Teacher model google/gemini-3.7-flash (via OpenRouter, reasoning.effort=low) Page render PyMuPDF at 200 DPI, grayscale JPEG q90 Sampling stratified —… See the full description on the dataset page: https://huggingface.co/datasets/kailasa-ngpt/gemini-3.7-flash-ocr-26-aug-2026.

sourceHugging Faceotherupdated 1mo agoView on Hugging Face

0likes98downloads

Dataset Card

gemini-3.7-flash-ocr-26-aug-2026

Page-image → transcription pairs for finetuning a vision-language model to OCR Devanagari and Tamil printed books.

These labels are not human ground truth. They are the output of a teacher model, so its accuracy is the ceiling for anything trained on them.

Provenance


Teacher model	`google/gemini-3.7-flash` (via OpenRouter, `reasoning.effort=low`)
Page render	PyMuPDF at 200 DPI, grayscale JPEG q90
Sampling	stratified — every book contributes; ~12 pages/Devanagari book, ~45/Tamil book
Built	2026-08-26

Sampling is per-book rather than exhaustive because diversity lives on the book axis (872 books), not the page axis — pages within a book share a typeface and scan run. Per-script budgets differ because the source corpus is ~95:5 Devanagari:Tamil, which would otherwise drown Tamil out.

	pages	books
train	12,508	832
validation	609	40
total	13,117	872

Script mix: {'Devanagari': 8783, 'Tamil': 4266, 'Telugu': 68} Characters: 21,379,152

Split by book, not by page. Pages within a book are near-duplicates; a page-level split would leak near-twins into validation and overstate the score. Validation measures generalisation to unseen books.

Fields

—image — the page, grayscale JPEG
—text — transcription (the training target)
—script — Devanagari | Tamil | Telugu
—book_type, book, serial_id, page
—pdf_drive_id, page_index, dpi — recipe to re-render the page from source
—recovered — true if the page needed a retry after a blank teacher response

Prompt used

You are an expert OCR transcriber for devanagari_letters texts printed in Devanagari script.
Book: <book title> | Type: sanskrit_scripture
Layout: single-column

Transcribe the ENTIRE page image verbatim, exactly as printed.

RULES -- FOLLOW EXACTLY:
1. Output the page text and NOTHING else. No JSON, no markdown, no LaTeX,
   no commentary. Never use \text{}, \underbrace{}, $...$ or any other
   markup -- transcribe accents, subscripts and musical notation as the
   plain Unicode characters they are printed as, and if a mark cannot be
   represented in Unicode, omit it rather than describing it.
2. Transcribe in natural reading order, top to bottom, as laid out on the page.
3. Preserve line breaks as printed: one printed line = one output line.
4. Transcribe EVERY character, including running headers, page numbers,
   verse numbers, footnotes, marginalia and catchwords -- each in the
   position where it actually appears on the page.
5. Do NOT translate, summarise, paraphrase, reorder or explain anything.
6. Do NOT invent headings, labels or numbers that are not printed on the page.
7. Preserve every diacritical mark exactly. Write Unicode directly as UTF-8,
   never as \u escape sequences.
8. If the page carries no text at all (blank, plate, cover), output exactly: [BLANK]

Known defects (measured, not estimated)


Teacher blank rate	~10% of calls return an empty response. 416 pages recovered by retry; any that stayed blank were dropped.
Markdown	The teacher wraps grids in markdown. Non-page decoration (headings, emphasis, rules) was stripped; table structure was deliberately kept so a model trained here emits a table when the page has one. 523 pages contain tables.
Degenerate output	LaTeX markup, decoder loops and runaway lines removed by QC (~0.4% of pages).
Script labels	Three books were mislabelled by upstream detection and relabelled from their actual text.
Numerals	The teacher occasionally confuses old-style Devanagari ९/६. Not corrected.
Normalisation	The teacher silently regularises archaic forms and printed typos. This is not a literal-glyph corpus.

Baseline