SoMarkAI/DocParsingBench
DocParsingBench is a document intelligence benchmark of 1,400 images, systematically collected and annotated from real business workflows. It is the first dataset to systematically catalogue the document elements most frequently encountered in enterprise settings, covering five major domains: finance, legal, scientific research, manufacturing, and education. 🆕 Latest Updates [2026.04.17] DocParsingBench evaluation toolkit release. Unified scoring is now available for the… See the full description on the dataset page: https://huggingface.co/datasets/SoMarkAI/DocParsingBench.
5365
1---2license: odc-by3tags:4 - document-parsing5 - document-intelligence6 - OCR Bench7 - Doc Parsing8 - OCR9 - pdf10 - markdown11---12 13DocParsingBench is a document intelligence benchmark of 1,400 images, systematically collected and annotated from real business workflows. It is the first dataset to systematically catalogue the document elements most frequently encountered in enterprise settings, covering five major domains: finance, legal, scientific research, manufacturing, and education.14 15[](https://huggingface.co/datasets/SoMarkAI/DocParsingBench)16[](https://modelscope.cn/datasets/SoMark/DocParsingBench)17[](LICENSE)18[]()19 20 21---22## 🆕 Latest Updates23 24**[2026.04.17]** [DocParsingBench](https://github.com/SoMarkAI/DocParsingBench) evaluation toolkit release. Unified scoring is now available for the three core elements of document parsing — **text, formulas, and tables** — together with **batch CLI evaluation, segment-level matching, visualization analysis, and leaderboard generation**. 📊25 26**[2026.03.09]** DocParsingBench dataset release. The first document parsing dataset built for real-world industry scenarios, covering finance, legal, scientific research, manufacturing, and education. Now available on [Hugging Face](https://huggingface.co/datasets/SoMarkAI/DocParsingBench) and [ModelScope](https://modelscope.cn/datasets/SoMark/DocParsingBench)! 🔥🔥🔥27 28---29 30## 🎯 Key Highlights31 32### ✅ 1. Five Real-World Industry Domains33 34Samples were drawn from **real business workflows**, preserving **scan noise, stamp occlusion, and blurred characters**:35 36| Domain | Document Types | Characteristics |37|--------|----------------|-----------------|38| **Finance** | Brokerage research reports, listed-company annual reports, prospectuses | Multi-column tables, stamped PDF scans, complex tables |39| **Legal** | Legal documents, contract clauses, industry standards | Standardized headers and footers, dense footnotes, cross-referenced citations |40| **Scientific Research** | Academic papers, programming textbooks, full-text patents | Mixed two- and three-column layouts, formula-heavy text, code blocks, chemical formulas |41| **Manufacturing** | Operating SOPs, forms and receipts | Blurry scans, handwritten fill-ins, QR codes / barcodes |42| **Education** | Textbooks across English, chemistry, and mathematics | Chemical structures, reaction equations, multiple-choice options, fill-in-the-blanks |43 44 45### ✅ 2. Full Coverage of Complex Layouts46 47- **Single-column**: textbooks, legal documents48- **Two-column**: academic papers, some research reports49- **Three or more columns**: brokerage financial reports, data reports50- **Mixed layouts**: interleaved text and figures, tables spanning columns, nested sidebars51 52---53 54## 📊 Dataset Composition55 56| Dimension | Category |57|-----------|----------|58| **Total samples** | 1,400 pages |59| **Languages** | Chinese, English, Chinese-English mixed |60| **Industry distribution** | Finance / Legal / Scientific Research / Manufacturing / Education |61| **Layout distribution** | Single-column / Two-column / Three-column / Mixed |62| **Annotation format** | Markdown |63| **Chemistry annotation** | Follows the [SoMarkdown](https://github.com/SoMarkAI/SoMarkDown) specification, combining SMILES with LaTeX to fully render chemical structures |64 65---66 67## 🏗️ Annotation Philosophy68- **Fully human-annotated, without exception** — the "ground truth" of document parsing should not be something a model guesses. In practice, we found that errors from model pre-annotation are silently accepted by annotators: when shown an existing markdown draft, annotators tend to tweak rather than redraw, and the model's mistakes get frozen into the dataset. Every bounding box is drawn stroke by stroke; every character is typed by hand. It is not the fastest approach, but it is the one that yields the highest quality.69- This was the most time-consuming part of the project. We wrote detailed boundary rules for every class of document element, then pilot-annotated, iterated, and finalized them class by class.70- Every spec went through 3+ rounds of pilot annotation, team-wide training, and consistency testing, keeping inter-annotator disagreement on the same page below 5%. In total, 6 full batches were rejected and re-annotated, with an annotation-to-review time ratio of 1 : 0.8.71 72 73## 📈 Evaluation Benchmark74 75The companion [**industrial document parsing benchmark**](https://github.com/SoMarkAI/DocParsingBench) is released alongside this dataset — community submissions are welcome! 📊76 77 78## ❤️ Acknowledgements79 80This is work that demands extraordinary patience. A single complex page mixing chemical formulas and tables can take hours; one boundary disagreement can send an entire batch back for re-annotation, costing hundreds of person-hours. Behind every annotated page is a human eye parsing the document's structure and a human hand confirming the markdown. Thanks to every annotator who saw this through.81 82We also thank the [SoMarkdown](https://github.com/SoMarkAI/SoMarkDown) project for the chemical-structure annotation specification that let us express LLM-readable chemistry precisely.83 84---85 86## 📖 Citation87 88If you use DocParsingBench in your research, please cite it as follows:89 90```bibtex91@misc{DocParsingBench-2026,92 title={DocParsingBench},93 author={SoMark},94 year={2026},95 publisher={Hugging Face,ModelScope},96 howpublished={\url{https://modelscope.cn/datasets/SoMark/DocParsingBench}}97}98```99 100---101 102## 📜 License103 104This project is released under the **ODC-BY (Open Data Commons Attribution License)** and is open to both academic research and commercial use.105 106---107> 🌟 If you find this dataset useful, please consider giving us a ⭐ on ModelScope / Hugging Face!108 109 110#### How to Download111 112**🤗 Hugging Face**113 114```python115from datasets import load_dataset116ds = load_dataset("SoMarkAI/DocParsingBench")117```118 119**☁️ ModelScope**120 121```python122from modelscope.msdatasets import MsDataset123ds = MsDataset.load('SoMark/DocParsingBench')124```125 126**Git Clone**127 128```bash129# Hugging Face130git clone https://huggingface.co/datasets/SoMarkAI/DocParsingBench131 132# ModelScope133git clone https://www.modelscope.cn/datasets/SoMark/DocParsingBench.git134```135