datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
govdocs1-pdf-source
govdocs1: source PDF files
[!NOTE]
Converted versions of other document types (word, txt, etc) are available in this repo
This is ~220,000 open-access PDF documents (about 6.6M pages) from the dataset govdocs1. It wants to be OCR'd.
Uploaded as tar file pieces of ~10 GiB each due to size/file count limits with an index.csv covering details
5,000 randomly sampled PDFs are available unarchived in sample/. Hugging Face supports previewing these in-browser, for example this one… See the full description on the dataset page: https://huggingface.co/datasets/BEE-spoke-data/govdocs1-pdf-source.Techie_Raw_PDFUN_Historical_PDF_Article_Text_Corpus
python
dataset = load_dataset("ranWang/UN_Historical_PDF_Article_Text_Corpus", split="train")
or
dataset = load_dataset("ranWang/UN_Historical_PDF_Article_Text_Corpus", split="randomTest")
lang_list = ["ar", "en", "es", "fr", "ru", "zh"]
for row in dataset:
# 获取pdf文章内容
for lang in lang_list:
# type == str
lang_match_file_content = row[lang]
# 如果按页分割
lang_match_file_pages_content = lang_match_file_content.split("\n----\n")
pdfa-eng-wds-filteredpdf_benchFor benchmarking pdf operations. A small subset of pdfa-eng-wds.
olmocr_science_pdfs-literaturehttps://huggingface.co/datasets/allenai/dolma3_pool/tree/main/data/olmocr_science_pdfs-literature
pdf-parsing-bench-resultsICLR-pdfs
Dataset Card for "ICLR-pdfs"
More Information needed
pdf_images_filtered
Image Dataset with Parquet Format
This dataset contains images with their IDs in parquet format for efficient loading.
Dataset Structure
Each configuration (language) contains:
image: PIL Image object
id: String identifier
Languages
Arabic (ar): 955 images
Bengali (bn): 932 images
German (de): 940 images
English (en): 932 images
Spanish (es): 951 images
French (fr): 947 images
Gujarati (gu): 949 images
Hindi (hi): 891 images
Italian (it): 1,005 images… See the full description on the dataset page: https://huggingface.co/datasets/v1v1d/pdf_images_filtered.unboxgov-khmdhs-pdfs
KHMDHS Contract Attachment PDFs
Per-contract attachment PDFs from the Greek public-procurement registry
KHMDHS, downsampled with Ghostscript /ebook (~150 DPI) for fast viewing
and compact storage. Files are packed into per-month SQLite shards
(pdfs/<YYYY-MM>.sqlite, one row per ΑΔΑΜ, keyed by referenceNumber).
These are lossy, downsampled copies. The authoritative originals remain at
KHMDHS: https://cerpp.eprocurement.gov.gr/khmdhs-opendata/contract/attachment/{ΑΔΑΜ}.
This… See the full description on the dataset page: https://huggingface.co/datasets/vasilisplavos/unboxgov-khmdhs-pdfs.idl-wds-pdfs-images-shuffledcc_main_2024_51_links_pdf_urlocr-pdf-degraded
OCR-PDF-Degraded Dataset
Overview
This dataset contains synthetically degraded document images paired with their ground truth OCR text. It addresses a critical gap in OCR model training by providing realistic document degradations that simulate real-world conditions encountered in production environments.
Purpose
Most OCR models are trained on relatively clean, perfectly scanned documents. However, in real-world applications, especially in the military/defense… See the full description on the dataset page: https://huggingface.co/datasets/racineai/ocr-pdf-degraded.GDP.pdf
GDP.pdf
GDP.pdf
GDP.pdf measures professional multimodal reasoning over the documents the economy actually runs on: dense reports, contracts, filings, and records in the messy real-world formats professionals work from. It was cited in Anthropic's Fable 5 and Mythos 5 model card.
What it tests
The benchmark contains 100 real-world prompts and PDFs pulled directly from professional workflows across ten domains: Finance, Healthcare, Legal, STEM/Research, Engineering, Construction… See the full description on the dataset page: https://huggingface.co/datasets/tolo6474/GDP.pdf.pdf_science_questions_verified_r1_traces__2_24_25
Dataset card for pdf_science_questions_verified_r1_traces__2_24_25
This dataset was made with Curator.
Dataset details
A sample from the dataset:
{
"url": "https://www.ttcho.com/_files/ugd/988b76_01ceeff230b24cbbb0125b2bfa3f3475.pdf",
"filename": "988b76_01ceeff230b24cbbb0125b2bfa3f3475.pdf",
"success": true,
"page_count": 37,
"page_number": 1,
"question_choices_solutions": "QUESTION: What is the identity of X in the reaction 14N + 1n \u2192… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations-dev/pdf_science_questions_verified_r1_traces__2_24_25.PDF_and_SCP_unfiltered_organic_chemistry_questionsOpenDoc-Pdf-Preview
OpenDoc-Pdf-Preview
OpenDoc-Pdf-Preview is a compact visual preview dataset containing 6,000 high-resolution document images extracted from PDFs. This dataset is designed for Image-to-Text tasks such as document OCR pretraining, layout understanding, and multimodal document analysis.
Dataset Summary
Modality: Image-to-Text
Content Type: PDF-based document previews
Number of Samples: 6,000
Language: English
Format: Parquet
Split: train only
Size: 606 MB
License: Apache… See the full description on the dataset page: https://huggingface.co/datasets/prithivMLmods/OpenDoc-Pdf-Preview.books4_pdfGDP.pdf
GDP.pdf
GDP.pdf
GDP.pdf measures professional multimodal reasoning over the documents the economy actually runs on: dense reports, contracts, filings, and records in the messy real-world formats professionals work from. It was cited in Anthropic's Fable 5 and Mythos 5 model card.
What it tests
The benchmark contains 100 real-world prompts and PDFs pulled directly from professional workflows across ten domains: Finance, Healthcare, Legal, STEM/Research, Engineering, Construction… See the full description on the dataset page: https://huggingface.co/datasets/Oitaaa/GDP.pdf.GDP.pdf
GDP.pdf
GDP.pdf measures professional multimodal reasoning over the documents the economy actually runs on: dense reports, contracts, filings, and records in the messy real-world formats professionals work from. It was cited in Anthropic's Fable 5 and Mythos 5 model card.
What it tests
The benchmark contains 100 real-world prompts and PDFs pulled directly from professional workflows across ten domains: Finance, Healthcare, Legal, STEM/Research, Engineering… See the full description on the dataset page: https://huggingface.co/datasets/cloudybean/GDP.pdf.wiki_synth_qa_redstone_dclmknw_fwedu_pdf_mixpdf-ocr-datasetCC-MAIN-pdf-urlun_pdf_text_data_test
Dataset Card for "un_pdf_text_data_test"
More Information needed
GDP.pdf
GDP.pdf
A benchmark for evaluating PDF parsing and document understanding systems.
[!WARNING]
Evaluation use only — do not train on this data.
This dataset is published as a held-out benchmark. Including it (or any
derivative of it) in model training or fine-tuning data contaminates the
benchmark and invalidates results. The data is provided in a single test
split for this reason; there is intentionally no train split.
If you operate a web crawler or assemble training corpora… See the full description on the dataset page: https://huggingface.co/datasets/sleepyheeler/GDP.pdf.CUAD_v1_Contract_Understanding_PDF
Dataset Card for Contract Understanding Atticus Dataset (CUAD) PDF
This dataset contains the PDFs and the full text of 509 commercial legal contracts from the original CUAD dataset. One of the original 510 contracts was removed due to being a scanned copy.
The extracted text was cleaned using clean-text.
The PDFs were encoded in base64 and added as the pdf_bytes_base64 feature.
You can easily and quickly load it:
dataset = load_dataset("dvgodoy/CUAD_v1_Contract_Understanding_PDF")… See the full description on the dataset page: https://huggingface.co/datasets/dvgodoy/CUAD_v1_Contract_Understanding_PDF.HARMLESS_Synthetic_Injected_PDFs_EDA
Injected PDFs - EDA and Evaluation Corpus
This repository holds the exploratory data analysis for a project on detecting harmless-but-real
attack payloads injected into PDF files, together with the dataset that analysis produced.
The project has two halves, both in the notebook Final_project_V7_EDA.ipynb:
Question
Input
Part 1
Is our synthetic corpus a stand-in for real malware, or is it something else?
The published CIC feature table (11,126 x 34)
Part 2
Is our… See the full description on the dataset page: https://huggingface.co/datasets/Cyber-security-final-project/HARMLESS_Synthetic_Injected_PDFs_EDA.pdf_science_questions_verifiable_r1_traces__2_24_25
Dataset card for pdf_science_questions_verifiable_r1_traces__2_24_25
This dataset was made with Curator.
Dataset details
A sample from the dataset:
{
"url": "https://www.ttcho.com/_files/ugd/988b76_01ceeff230b24cbbb0125b2bfa3f3475.pdf",
"filename": "988b76_01ceeff230b24cbbb0125b2bfa3f3475.pdf",
"success": true,
"page_count": 37,
"page_number": 1,
"question_choices_solutions": "QUESTION: What is the identity of X in the reaction 14N + 1n \u2192… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations-dev/pdf_science_questions_verifiable_r1_traces__2_24_25.UPRPRC_pdffiles_from_UNThis datasets contains all the raw PDF file crawl from United Nations Digital Library, produced by https://github.com/mnbvc-parallel-corpus-team/UPRPRC/blob/v2_record_spider/scripts/v4_list2doc.py, using the index in https://huggingface.co/datasets/bot-yaya/documents.un.org_search_result.
If you are writing spider script to download all these files, you can do increment download based on this dataset.
Our UPRPRC project: https://github.com/mnbvc-parallel-corpus-team/UPRPRC
Attention: Record… See the full description on the dataset page: https://huggingface.co/datasets/bot-yaya/UPRPRC_pdffiles_from_UN.olmocr_science_pdfs-software_developmenthttps://huggingface.co/datasets/allenai/dolma3_pool/tree/main/data/olmocr_science_pdfs-software_development