datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
VDR_MEGA_2
VDR_MEGA_2
Dataset Summary
VDR_MEGA_2 is a high-quality multimodal dataset created through the merge of multiple domain-specific datasets with enhanced data processing techniques. This dataset represents our most refined approach to multimodal data generation, incorporating filtering algorithms and improved AI-assisted content generation to deliver superior quality for RAG, DSE, question answering, document search, and vision-language model training tasks.… See the full description on the dataset page: https://huggingface.co/datasets/racineai/VDR_MEGA_2.RefSpatial
⚠️ Warning: The Dataset Viewer and Data Studio above are for display only. They show just 800 samples from the full RefSpatial dataset, taken from the "SubsetVisualization" folder in Hugging Face ".parquet" format.
ℹ️ Info: The full raw dataset (~357GB) is available in non-HF formats (e.g., images, depth maps, JSON files).
RefSpatial: A Large-scale Dataset for teaching a general VLM to achieve spatial referring with reasoning… See the full description on the dataset page: https://huggingface.co/datasets/JingkunAn/RefSpatial.ROPE
Dataset Card for ROPE
The dataset used in this study is designed to evaluate and analyze multi-object hallucination by leveraging existing panoptic segmentation datasets. Specifically, it includes data from MSCOCO-Panoptic and ADE20K, ensuring access to diverse objects and their instance-level semantic annotations.
For more information, please visit Multi-Object Hallucination.
Dataset Construction
The dataset is divided into several subsets based on the distribution… See the full description on the dataset page: https://huggingface.co/datasets/sled-umich/ROPE.Edge-Agent-Reasoning-WebSearch-260K
Edge Agent Reasoning WebSearch 260K
Abstract
The Edge-Agent-Reasoning-WebSearch-260K dataset is a massive, synthetically expert-engineered corpus of over 700 Million tokens, designed to train small, local models (SLMs) and edge-deployed agents in advanced problem deconstruction and self-aware reasoning.
Rather than training a model to execute instructions directly—which often leads to hallucinations when context is missing—this dataset trains a model to act as a… See the full description on the dataset page: https://huggingface.co/datasets/yatin-superintelligence/Edge-Agent-Reasoning-WebSearch-260K.t2-ragbench
Dataset Card for T2-RAGBench
Project Page | Paper | Code
IMPORTANT NOTICE:
We deleted VQAonBD from the dataset due to low quality of the question reformulations. If you still want to use it you will find the data in the previous commit history.
Dataset Description
Dataset Summary
T2-RAGBench is a benchmark dataset designed to evaluate Retrieval-Augmented Generation (RAG) on financial documents containing both text and tables. It consists of 23,088… See the full description on the dataset page: https://huggingface.co/datasets/G4KMU/t2-ragbench.Omnimodal-Agent-SFT-2K
OmniGAIA: Omni-Modal General AI Assistant Benchmark
📄 Paper
•
💻 Code & Demo
•
🤗 Dataset & Model
•
📈 Leaderboard
This dataset contains omni-modal agent supervised fine-tuning (SFT) trajectories in the LlamaFactory SFT data format. You can directly follow LlamaFactory's instructions to fine-tune your omni-modal LLMs.OmniGAIA is a benchmark for Omni-Modal General AI Assistants that jointly reason over vision, audio, and language with external tools. It is… See the full description on the dataset page: https://huggingface.co/datasets/RUC-NLPIR/Omnimodal-Agent-SFT-2K.JieZi
JieZi (解字)
💻 Project ·
📦 Dataset ·
🌐 Demo
JieZi (解字) is a large-scale, expert-audited visual question answering (VQA) dataset dedicated to ancient Chinese character exegesis. It pairs high-quality character glyph images with fine-grained expert annotations across nine paleographic tasks—including headword recognition, etymology, structural analysis, glyph evolution, and component function—providing a rigorous benchmark for multimodal… See the full description on the dataset page: https://huggingface.co/datasets/Ran0/JieZi.jee-neet-benchmark
JEE/NEET LLM Benchmark Dataset
🏆 View the live leaderboard → — interactive results across JEE Advanced, JEE Main & NEET, with open/closed-weight badges, contamination flags, and per-run cost.
A benchmark for evaluating vision-capable LLMs on Indian competitive exam questions (JEE Advanced & NEET). Each question is the original exam image; models answer via the OpenRouter API and are scored with authentic, exam-specific marking schemes — including partial credit for JEE… See the full description on the dataset page: https://huggingface.co/datasets/Reja1/jee-neet-benchmark.RPC-Bench
RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension
🌐 Project Page •
💻 GitHub •
📖 Paper
RPC-Bench is a fine-grained benchmark for research paper comprehension. It is built from review-rebuttal exchanges of high-quality academic papers and supports both text-only and visual evaluation through complementary paper representations.
Data Structure
RPC-Bench is organized into train, dev, and test subsets. Split assignments… See the full description on the dataset page: https://huggingface.co/datasets/zai-org/RPC-Bench.Curr-ReFT-data
Curr-ReFT-data
[📂 GitHub][📝 Paper]
[🤗 HF Dataset] [🤗 HF-Model: Curr-ReFT-3B]
[🤗 HF-Model: Curr-ReFT-7B]
Dataset Overview
Curr-ReFT-data contains training data for both stages of the Curr-ReFT methodology. The proposed Curr-ReFT post-training paradigm consists of two consecutive training stages: 1. Curriculum Reinforcement Learning: Gradually increasing task difficulty through reward mechanisms that match task complexity. 2. Rejected Sample based… See the full description on the dataset page: https://huggingface.co/datasets/ZTE-AIM/Curr-ReFT-data.Robot-EQ
RobotEQ-Data
Official dataset release for RobotEQ.
Evaluation & Scripts
For inference scripts, evaluation scripts, and data production tooling, see the RobotEQ code repository.
Dataset Statistics
Item
Count
Behavior judgment scenarios (synthetic)
1,812
Behavior judgment scenarios (real POV)
223
Behavior judgment scenarios (total)
2,035
Behavior judgment behavior annotations
3,171
Spatial grounding questions
825… See the full description on the dataset page: https://huggingface.co/datasets/Tongji-Emotion/Robot-EQ.RefSpatial-Bench
🎉 RefSpatial-Expand-Bench is officially released!
The new version not only extends indoor scenes (e.g., factories, stores), but also introduces brand-new outdoor scenarios (e.g., streets, parking lots) — enabling more comprehensive evaluation of spatial referring tasks.
👉 Try it now: RefSpatial-Expand-Bench
🏆 The paper associated with this benchmark, RoboRefer, has been accepted to NeurIPS 2025!
Thank you all for your attention and support! 🙌… See the full description on the dataset page: https://huggingface.co/datasets/BAAI/RefSpatial-Bench.VisualWebInstruct-Recall
Introduction
This is the dataset recalled from Google Search from the seed images.
Links
Github|
Paper|
Website
Citation
@article{visualwebinstruct,
title={VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search},
author = {Jia, Yiming and Li, Jiachen and Yue, Xiang and Li, Bo and Nie, Ping and Zou, Kai and Chen, Wenhu},
journal={arXiv preprint arXiv:2503.10582},
year={2025}
}
RoboSpatial-Home
RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics
🌐 Homepage | 📖 arXiv | 🛠️ Data Gen | 🧪 Eval Code
🔥 Core spatial understanding benchmark used by Qwen3-VL and Gemini Robotics!
⚠️ Important Note
Annotation Correction (08/14/25): In the context category, the microwave question was corrected from “below” to “above” to fix an annotation error.Annotation Correction (05/13/26): In the configuration category, the… See the full description on the dataset page: https://huggingface.co/datasets/chanhee-luke/RoboSpatial-Home.realms-of-omnarai
The Realms of Omnarai
Where frontier intelligences actually disagree — verbatim, attributed, traceable. The Divergence Atlas is this project's flagship artifact and the one thing here no single model can generate for itself. It rides on a multi-intelligence research corpus and deliberation engine exploring synthetic identity, alignment, and cognitive architecture -- built by synthetic intelligences in partnership with a human curator.
The Atlas is the payoff; the Memory Engine… See the full description on the dataset page: https://huggingface.co/datasets/TheRealmsOfOmnarai/realms-of-omnarai.vsr_random
VSR: Visual Spatial Reasoning
This is the random set of VSR: Visual Spatial Reasoning (TACL 2023) [paper].
Usage
from datasets import load_dataset
data_files = {"train": "train.jsonl", "dev": "dev.jsonl", "test": "test.jsonl"}
dataset = load_dataset("cambridgeltl/vsr_random", data_files=data_files)
Note that the image files still need to be downloaded separately. See data/ for details.
Go to our github repo for more introductions.
Citation
If you find VSR… See the full description on the dataset page: https://huggingface.co/datasets/cambridgeltl/vsr_random.MMSI-Bench
MMSI-Bench
This repo contains evaluation code for the paper "MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence"
🌐 Homepage | 🤗 Dataset | 📑 Paper | 💻 Code | 📖 arXiv
🔔News
🔥[2025-10-23]: We added the normalized human response time for each MMSI-Bench sample and its difficulty level to our dataset on Hugging Face.
🔥[2025-06-18]: MMSI-Bench has been supported in the LMMs-Eval repository.
✨[2025-06-11]: MMSI-Bench was used for evaluation in the… See the full description on the dataset page: https://huggingface.co/datasets/RunsenXu/MMSI-Bench.VDR_Nuclear
VDR_Nuclear - Overview
Dataset Summary
VDR_Nuclear is a curated multimodal dataset focused on nuclear technical documents, regulations, and legal frameworks. It combines text and image data extracted from real scientific and regulatory PDFs to support tasks such as RAG DSE, question answering, document search, and vision-language model training.
Dataset Creation
This dataset was created using our open-source tool VDR_pdf-to-parquet.
Nuclear-related PDFs were… See the full description on the dataset page: https://huggingface.co/datasets/racineai/VDR_Nuclear.VDR_Qualitative
VDR_Qualitative
Dataset Summary
VDR_Qualitative is a high-quality multimodal dataset created through the merge of multiple domain-specific datasets with enhanced data processing techniques. This dataset represents our most refined approach to multimodal data generation, incorporating filtering algorithms and improved AI-assisted content generation to deliver superior quality for RAG, DSE, question answering, document search, and vision-language model training tasks.… See the full description on the dataset page: https://huggingface.co/datasets/racineai/VDR_Qualitative.refute
Can AI read new science honestly?
Models can sound convincing while misreading a result or expressing more confidence than the evidence deserves. That matters when people use them to summarize papers, compare studies, or decide what to investigate next.
REFUTE tests whether a model knows the finding, spots quiet flaws, names what would overturn a claim, and matches its confidence to the evidence.
Truth Score is the main result. It combines factual accuracy, flaw… See the full description on the dataset page: https://huggingface.co/datasets/BGPT-OFFICIAL/refute.ttrpg-rpg-fandom-com-en
ttrpg-rpg-fandom-com-en (RPG Fandom EN Dataset)
[Russian version below / Русская версия ниже]
Description
This dataset contains a complete dump of the English rpg.fandom.com wiki, converted to clean Markdown. It is designed for RAG (Retrieval-Augmented Generation), LLM fine-tuning, and research.
Structure
markdown/: Cleaned documents with metadata.
indexes/documents.jsonl: Global document registry.
indexes/chunks.jsonl: Semantic fragments for… See the full description on the dataset page: https://huggingface.co/datasets/exnihilum/ttrpg-rpg-fandom-com-en.VDR_Renewable_Regulation
VDR_Renewable_Regulation - Overview
Dataset Summary
VDR_Renewable_Regulation is a curated multimodal dataset focused on renewable energy technical documents, regulations, and legal frameworks. It combines text and image data extracted from real scientific and regulatory PDFs to support tasks such as RAG DSE, question answering, document search, and vision-language model training.
Dataset Creation
This dataset was created using our open-source tool… See the full description on the dataset page: https://huggingface.co/datasets/racineai/VDR_Renewable_Regulation.human-coherence-preferences-images
Rapidata Image Generation Coherence Dataset
This dataset was collected in ~4 Days using the Rapidata Python API, accessible to anyone and ideal for large scale data annotation.
Explore our latest model rankings on our website.
If you get value from this dataset and would like to see more in the future, please consider liking it.
Overview
One of the largest human annotated coherence datasets for text-to-image models, this release contains over 1,200,000 human… See the full description on the dataset page: https://huggingface.co/datasets/Rapidata/human-coherence-preferences-images.R-Bench
R-Bench
Introduction
R-Bench is a graduate-level multi-disciplinary benchmark for evaluating the complex reasoning capabilities of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs). R stands for Reasoning.
According to statistics on R-Bench, the benchmark spans 19 departments, including mathematics, physics, biology, computer science, and chemistry, covering over 100 subjects such as Inorganic Chemistry, Chemical Reaction Kinetics, and… See the full description on the dataset page: https://huggingface.co/datasets/R-Bench/R-Bench.OmniGAIA
OmniGAIA: Omni-Modal General AI Assistant Benchmark
📄 Paper
•
💻 Code & Demo
•
🤗 Dataset & Model
•
📈 Leaderboard
OmniGAIA is a benchmark for Omni-Modal General AI Assistants that jointly reason over vision, audio, and language with external tools. It is designed to evaluate long-horizon, multi-hop, open-form problem solving in realistic settings rather than short perception-only QA.
Benchmark Construction
The OmniGAIA construction… See the full description on the dataset page: https://huggingface.co/datasets/RUC-NLPIR/OmniGAIA.human-alignment-preferences-images
Rapidata Image Generation Alignment Dataset
This dataset was collected in ~4 Days using the Rapidata Python API, accessible to anyone and ideal for large scale data annotation.
Explore our latest model rankings on our website.
If you get value from this dataset and would like to see more in the future, please consider liking it.
Overview
One of the largest human annotated alignment datasets for text-to-image models, this release contains over 1,200,000 human… See the full description on the dataset page: https://huggingface.co/datasets/Rapidata/human-alignment-preferences-images.RICO-ScreenQA
Dataset Card for ScreenQA
Question answering on RICO screens: google-research-datasets/screen_qa.
Citation
BibTeX:
@misc{hsiao2024screenqa,
title={ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots},
author={Yu-Chung Hsiao and Fedir Zubach and Maria Wang and Jindong Chen},
year={2024},
eprint={2209.08199},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
rico_refexp_combined
Dataset Card for "rico_refexp_combined"
This dataset combines the crowdsourced RICO RefExp prompts from the UIBert dataset and the synthetically generated prompts from the seq2act dataset.
RiOSWorld
News
2025-05-31: We released our paper, environment and benchmark, and project page. Check it out!
Download and Setup
# DownLoad the RiOSWorld risk examples
dataset = load_dataset("JY-Young/RiOSWorld", split='test')
The environmental risk examples require specific configuration. For specific configuration processes, please refer to: https://github.com/yjyddq/RiOSWorld
Data Statistics
Topic Distribution of User Instruction… See the full description on the dataset page: https://huggingface.co/datasets/JY-Young/RiOSWorld.Edge-Agent-Reasoning-WebSearch-260K
Edge Agent Reasoning WebSearch 260K
Abstract
The Edge-Agent-Reasoning-WebSearch-260K dataset is a massive, synthetically expert-engineered corpus of over 700 Million tokens, designed to train small, local models (SLMs) and edge-deployed agents in advanced problem deconstruction and self-aware reasoning.
Rather than training a model to execute instructions directly—which often leads to hallucinations when context is missing—this dataset trains a model to act as a… See the full description on the dataset page: https://huggingface.co/datasets/BlueIsGreen/Edge-Agent-Reasoning-WebSearch-260K.