shahriarhd/persian-official-letters-synth

Persian Official Letters (Synthetic)

Draft dataset card. Copy this file to the root of the export tree and fill in the dataset_info / configs blocks from dataset_info.json. See docs/publishing_to_huggingface.md for the export and upload procedure. Replace <owner> in FA_OCR_HF_REPO_ID before pushing.

A 10,000-letter synthetic corpus of Persian administrative correspondence, rendered as scanned-looking documents with full layout ground truth: page images, per-region bounding boxes, region text, reading order, and structured business fields.

The corpus exists to fine-tune a vision-language OCR model on Persian official letters — a domain where clean public data is scarce and where the layout (letterhead, metadata block, subject, body, signature, stamp) carries as much meaning as the characters.

Dataset at a glance

from datasets import load_dataset

# default config is `images`
ds = load_dataset("<owner>/persian-official-letters-synth")
page = ds["train"][0]
page["image"].save("page.png")     # decoded PIL image
print(page["text"][:200])

# other artifacts by config name
anns = load_dataset("<owner>/persian-official-letters-synth", "annotations", split="train")
pdfs = load_dataset("<owner>/persian-official-letters-synth", "pdf", split="train")

Configs

Each config is a separate artifact family with its own shards, so any one can be loaded without the others.

Splits: train 90% / test 10%, assigned by hashing source_sample_id so all pages and re-renders of one letter stay on the same side of the split.

How the data was made

Three stages, each recorded so a sample is reproducible from its seed.

1. Text generation. Persian letter bodies were generated with nvidia/nemotron-3-nano-30b-a3b (temperature 0.4, top_p 0.7, max 8192 tokens, seed 1337), conditioned on a real company record drawn from a logo corpus. Each letter carries a subject, a Jalali date, a letter number, optionally a national ID and registration number, an attachments list, a signatory title and contact info. The generation model and its parameters are stored in each plaintext record's generation block.

2. Layout and rendering. Letters are laid out and printed to an HTML page in Chromium, then rasterised and printed to PDF. 12 Persian font families are embedded as @font-face data URIs, so output does not depend on host fonts. Layout draws from weighted distributions: 92% of letters have a letterhead, 78% a footer, 45% a religious opener (بسمه تعالی), 34% an underlined subject, ~10% a table or chart.

3. Material realism. Composited on top of the render:

• —Paper — warm/cool off-white tints; 49% of letters carry a real flat-fielded paper scan from a texture corpus; the other 51% render clean.
• —Signature — 7,066 letters carry a handwritten signature taken from the UTSig Persian signature corpus. The scan's white paper is discarded and only the ink survives as an alpha channel, recoloured (mostly blue-black), tilted ±7°, and allowed to overlap the signatory's title.
• —Stamp (مهر) — 5,602 letters carry a company seal: the brand logo inside a geometric border (circle, double circle, oval, rounded square, square, hexagon) ringed with Persian text, in office blue, red or violet, with turbulence-driven erosion for a pressed look.
• —Scanner simulation — applied to ~85% of samples: rotation, perspective, page warp, barrel distortion, uneven illumination, vignette, ink bleed, JPEG artefacts, Gaussian noise, dust specks, and occasional downsample-then-upscale, greyscale or 1-bit thresholding. Every applied value is recorded in the annotation's scan block.

Fields

Every config repeats the same ~30-column sample header (sample_id, source_sample_id, split, letter_genre, has_table, has_chart, has_signature, has_stamp, overflowed, page_count, width, height, dpi, page_size, orientation, …), so a consumer who loads only one family still gets the filter columns without a join.

The annotations config carries the full structure described in annotations/README.md: a pages[] array, each with width, height, image_path and an elements[] array of {id, type, bbox, text, reading_order, content?}. bbox is [x0, y0, x1, y1] in device pixels, origin top-left, indexing its page image directly.

Element types span six page regions — header (logo, header_bar, header_company_name, header_slogan, religious_opener), metadata (metadata_block, metadata_field), frame (sender, recipient, subject), body (paragraph, heading, list_item, table, table_cell, chart, attachments_block), closing (signature_block, signature_image, signatory_name, signatory_title, stamp) and footer (footer_bar, footer_field, page_number).

Intended uses

Use it for: fine-tuning and evaluating Persian document OCR and layout analysis; benchmarking reading-order reconstruction on RTL documents; training region detection for administrative letter structure; data augmentation alongside real scanned letters.

Out of scope: the corpus is synthetic. It should not be treated as a sample of real Iranian correspondence for any statistical, demographic or policy-related claim. Stamps and signatures are composited for research and testing only and carry no evidentiary value.

Limitations

• —Synthetic text. Letter bodies are model-generated Persian. Fluency is high but the text is not produced by human authors, and rare real-world phrasings, OCR-hostile names, and genuine administrative jargon are under-represented relative to a real corpus.
• —Two- to six-page letters only. Long documents (contracts, reports) are not represented.
• —A single page-image family. The corpus varies paper, lighting and noise heavily, but it does not cover multi-column layouts, forms with ruled grids, or photographed documents at extreme angles.
• —Digital-native artefacts. Despite the scan simulation, pages originate as a Chromium render — glyph shapes are clean font outlines, not degraded print.
• —Truncation. 167 letters had bodies long enough to exceed the 6-page ceiling and were truncated; they are flagged overflowed: true and should be filtered out of text tasks.
• —Persian-Indic digits only. Numerals are ۰-۹, and dates use the Jalali calendar with Persian month names; expect to normalise for tasks needing ASCII digits or Gregorian dates.

Licensing and attribution

Declared `cc-by-nc-4.0`, not cc-by-4.0. The pages composite third-party brand marks into generated seals and embed real people's handwriting from UTSig; both are research and non-commercial material. A permissive declaration on the container would contradict the terms of what it contains.

• —UTSig — Persian handwritten signature corpus, used for signature ink. Redistribution is governed by the original dataset's terms and its paper; cite the UTSig authors when using this corpus.
• —Logo corpus (thesvg) — 4,888 SVG marks and 4,888 PNG rasters republished as attribution/attribution.csv, one row per file with File_Name, Source, License, Author, Original_File. Licences are CC0-1.0 (6,556 rows), MIT (2,502) and Apache-2.0 (718). Keep this file with any redistribution — it is the attribution obligation for the brand marks that appear inside the letterheads and stamps.
• —Letter text — generated with nvidia/nemotron-3-nano-30b-a3b; credit the model when redistributing.
• —Every letter is synthetic, and every logo and stamp is present for research and testing only.

Citation

@misc{persian_official_letters_synth,
  title  = {Persian Official Letters (Synthetic)},
  author = {<owner>},
  year   = {2025},
  note   = {Synthetic Persian administrative correspondence with full layout
            annotations, 10,000 letters / 26,848 pages},
  url    = {https://huggingface.co/datasets/<owner>/persian-official-letters-synth}
}

If you use the signature material, also cite the UTSig dataset.

Files

.
├── README.md                  # this card
├── dataset_info.json          # features, per-split counts and sizes
├── metadata.csv               # flat per-page index (no pyarrow needed)
├── splits.csv                 # sample_id, source_sample_id, split
├── attribution/
│   └── attribution.csv        # per-logo licences
└── data/
    ├── images/                # parquet shards — page rasters
    ├── plaintext/             # parquet shards — source letter text
    ├── markdown/              # parquet shards — letter as Markdown
    ├── pdf/                   # parquet shards — whole letter as PDF
    ├── annotations/           # parquet shards — full ground truth
    └── metadata/              # parquet shards — filter flags only

Schema reference: `docs/dataset_schema.md` · Publishing guide: `docs/publishing_to_huggingface.md` · Design decisions: `docs/decisions.md`