smartytrios/document_data_extractor

Dataset Title: OCR-to-JSON Information Extraction

Project Overview

This dataset is specifically designed for fine-tuning Large Language Models (LLMs) to perform structured data extraction from Optical Character Recognition (OCR) outputs. The primary objective is to convert raw, unstructured text strings—often containing noise, misalignments, and formatting inconsistencies—into valid, machine-readable JSON objects.

Dataset Specifications

Data Structure

Each entry in the dataset consists of a single JSON object per line with two primary fields:

• —`prompt` – The raw OCR text extracted from a document (e.g., invoices, receipts, or forms).
• —`completion` – The corresponding ground-truth information structured as a nested JSON dictionary.

Data Field Definitions

Prompt Field

Represents the "source" text.

• —Includes headers, dates, numerical values, and currency symbols as recognized by standard OCR engines.
• —May contain line breaks (\n) and varied spacing representative of physical document layouts.

Completion Field

Represents the "target" output.

Follows a strict schema containing keys such as:

• —invoice_number
• —date
• —vendor_name
• —total_amount
• —line_items (list of objects)

Usage Instructions

Requirements

• —Python 3.10+
• —datasets (Hugging Face)
• —transformers library

Loading the Dataset

from datasets import load_dataset

dataset = load_dataset("json", data_files="real_dataset.jsonl", split="train")
print(dataset[0])