Nishant2414/OCR-Synthetic-Multilingual-v1

OCR-Synthetic-Multilingual-v1

Overview

Large-scale synthetically generated OCR training dataset for multilingual text detection and recognition. The data was produced using a heavily modified and extended version of SynthDoG (Synthetic Document Generator), originally introduced in the Donut project by Kim et al.

This dataset was used to train **Nemotron OCR v2**, a state-of-the-art multilingual OCR model that is part of the NVIDIA NeMo Retriever collection.

Languages

Numbers in parentheses are the number of .h5 files per split.

Related Model

This dataset was created to train the detection, recognition, and relational components of **Nemotron OCR v2**. See the model card for architecture details, evaluation results, and usage instructions.

Directory Layout

OCR-Synthetic-Multilingual-v1/
├── en/
│   ├── train/
│   │   ├── train_000.h5
│   │   ├── train_001.h5
│   │   └── ...
│   ├── test/
│   │   └── ...
│   └── validation/
│       └── ...
├── ja/
│   └── ...
├── ko/
│   └── ...
├── ru/
│   └── ...
├── zh_hans/
│   └── ...
└── zh_hant/
    └── ...

Format — HDF5

Each .h5 file contains the following datasets (HDF5 terminology):

Annotation JSON Schema

Each entry in annotations is a JSON object:

{
  "word_bboxes": [
    {
      "text": "example word or phrase",
      "bbox": [x, y, w, h],
      "quad": [[x0,y0], [x1,y1], [x2,y2], [x3,y3]]
    }
  ],
  "line_bboxes": [
    {
      "text": "full line of text",
      "bbox": [x, y, w, h],
      "quad": [[x0,y0], [x1,y1], [x2,y2], [x3,y3]],
      "para_idx": 0,
      "line_idx": 0,
      "word_indices": [0, 1, 2]
    }
  ],
  "para_bboxes": [...],
  "relation_graph": [
    [[0], [1], [2]],
    [[3], [4]]
  ]
}

Bounding Box Levels

• —`word_bboxes` — One entry per word/phrase rendered as a single unit. Each contains text, an axis-aligned bbox [x, y, w, h], and a 4-point quad.
• —`line_bboxes` — One entry per text line. Includes all word_bboxes fields plus para_idx (paragraph index), line_idx (line index within the paragraph), and word_indices (indices into word_bboxes that compose this line).
• —`para_bboxes` — One entry per paragraph bounding box.
• —`relation_graph` — Nested list encoding reading order: relation_graph[para][sentence] gives a list of word/line indices belonging to that sentence within the paragraph.

Quad Vertex Convention

Quads are 4-point polygons stored as [[x0,y0], [x1,y1], [x2,y2], [x3,y3]] in clockwise order:

v0 -------- v1
|            |
v3 -------- v2

Loading Example

import h5py, io, json
from PIL import Image

with h5py.File("en/train/train_000.h5", "r") as f:
    img_bytes = f["images"][0]
    image = Image.open(io.BytesIO(img_bytes.tobytes())).convert("RGB")

    annotation = json.loads(f["annotations"][0])
    for line in annotation["line_bboxes"]:
        print(line["text"], line["quad"])

Per-Language Details

English (en)

Japanese (ja)

Korean (ko)

Russian (ru)

Chinese Simplified (zh_hans)

Chinese Traditional (zh_hant)

Acknowledgements

The synthetic data generation pipeline is based on SynthDoG from the Donut project, with substantial modifications to support additional languages, custom rendering effects, structured bounding-box annotations (word/line/paragraph levels with reading-order graphs), and HDF5 output.

Citation

If you use this dataset, please cite:

@misc{chesler2026ocr_synthetic_multilingual,
  title   = {{OCR-Synthetic-Multilingual-v1}},
  author  = {Chesler, Ryan},
  year    = {2026},
  publisher = {NVIDIA},
  url     = {https://huggingface.co/datasets/nvidia/OCR-Synthetic-Multilingual-v1},
  note    = {Synthetically generated multilingual OCR dataset built on a heavily modified SynthDoG pipeline}
}