datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ParseBench
ParseBench
Quick links: [🌐 Website] [📜 Paper] [💻 Code]
ParseBench is a benchmark for evaluating document parsing systems on real-world enterprise documents, with the following characteristics:
Multi-dimensional evaluation. The benchmark is stratified into five capability dimensions — tables, charts, content faithfulness, semantic formatting, and visual grounding — each with task-specific metrics designed to capture what agentic workflows depend on.
Real-world enterprise… See the full description on the dataset page: https://huggingface.co/datasets/llamaindex/ParseBench.ExtractBench
ExtractBench
Quick links: [🌐 Website] [📜 Paper] [💻 Code]
Given a document and a schema, a system returns structured data with evidence. The input is a full document, born-digital or scanned, and a schema written by the user. The output is a schema-valid JSON object, with the source page and a bounding box for each value as evidence. It must return correct, exhaustive values (including repeated records), correctly use null for absent information, and ground each extracted… See the full description on the dataset page: https://huggingface.co/datasets/llamaindex/ExtractBench.vdr-multilingual-train
Multilingual Visual Document Retrieval Dataset
This dataset consists of 500k multilingual query image samples, collected and generated from scratch using public internet pdfs. The queries are synthetic and generated using VLMs (gemini-1.5-pro and Qwen2-VL-72B).
It was used to train the vdr-2b-multi-v1 retrieval multimodal, multilingual embedding model.
How it was created
This is the entire data pipeline used to create the Italian subset of this dataset. Each step… See the full description on the dataset page: https://huggingface.co/datasets/llamaindex/vdr-multilingual-train.llamaindex-vdr-en-train-preprocessed
llamaindex-vdr-en-train-preprocessed
This dataset is a preprocessed English subset of llamaindex/vdr-multilingual-train, prepared for training multimodal Sentence Transformer embedding models on document screenshot retrieval.
Changes from the original dataset
The original llamaindex/vdr-multilingual-train dataset stores hard negatives as a list of ID strings that reference other rows. This dataset makes two key changes:
English only: Only the English subset (53,512… See the full description on the dataset page: https://huggingface.co/datasets/tomaarsen/llamaindex-vdr-en-train-preprocessed.llamaindex-vdr-images
LlamaIndex VDR Images
This dataset contains the document-page images used by lightonai/llamaindex-vdr-fine-tuning. The pair is a reformatted derivative of llamaindex/vdr-multilingual-train for multilingual retrieval fine-tuning.
Dataset structure
The train split contains:
Column
Type
Description
image_filename
string
Stable key used by the companion fine-tuning dataset.
image
image
Document-page image.
Load the dataset
from… See the full description on the dataset page: https://huggingface.co/datasets/lightonai/llamaindex-vdr-images.vdr-multilingual-test
Multilingual Visual Document Retrieval Benchmarks
This dataset consists of 15 different benchmarks used to initially evaluate the vdr-2b-multi-v1 multimodal retrieval embedding model. These benchmarks allow the testing of multilingual, multimodal retrieval capabilities on text-only, visual-only and mixed page screenshots.
Each language subset contains queries and images in that language and is divided into three different categories by the "pagetype" column. Each category contains… See the full description on the dataset page: https://huggingface.co/datasets/llamaindex/vdr-multilingual-test.filtered-llamaindex-with-translations