datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MathVista
Dataset Card for MathVista
Dataset Description
Paper Information
Dataset Examples
Leaderboard
Dataset Usage
Data Downloading
Data Format
Data Visualization
Data Source
Automatic Evaluation
License
Citation
Dataset Description
MathVista is a consolidated Mathematical reasoning benchmark within Visual contexts. It consists of three newly created datasets, IQTest, FunctionQA, and PaperQA, which address the missing visual domains and are tailored to evaluate logical… See the full description on the dataset page: https://huggingface.co/datasets/AI4Math/MathVista.BlueMO
BlueMO
🚀 BlueMO: A Comprehensive Collection of Challenging Mathematical Olympiad Problems from the Little Blue Book Series
BlueMO is a comprehensive and challenging dataset comprising mathematical olympiad problems paired with detailed solutions, meticulously curated from the esteemed "Little Blue Book" (小蓝书) series (Second Edition)—a vital resource for Chinese students training for national and international olympiad math competitions.Designed to advance and… See the full description on the dataset page: https://huggingface.co/datasets/math-ai/BlueMO.Curr-ReFT-data
Curr-ReFT-data
[📂 GitHub][📝 Paper]
[🤗 HF Dataset] [🤗 HF-Model: Curr-ReFT-3B]
[🤗 HF-Model: Curr-ReFT-7B]
Dataset Overview
Curr-ReFT-data contains training data for both stages of the Curr-ReFT methodology. The proposed Curr-ReFT post-training paradigm consists of two consecutive training stages: 1. Curriculum Reinforcement Learning: Gradually increasing task difficulty through reward mechanisms that match task complexity. 2. Rejected Sample based… See the full description on the dataset page: https://huggingface.co/datasets/ZTE-AIM/Curr-ReFT-data.MathVerse
Dataset Card for MathVerse
Dataset Description
Paper Information
Dataset Examples
Leaderboard
Citation
Dataset Description
The capabilities of Multi-modal Large Language Models (MLLMs) in visual math problem-solvingremain insufficiently evaluated and understood. We investigate current benchmarks to incorporate excessive visual content within textual questions, which potentially assist MLLMs in deducing answers without truly interpreting the input diagrams.
To… See the full description on the dataset page: https://huggingface.co/datasets/AI4Math/MathVerse.IneqMath
Solving Inequality Proofs with Large Language Models
Introduction
Inequality proving, crucial across diverse scientific and mathematical fields, tests advanced reasoning skills such as discovering tight bounds and strategically applying theorems. This makes it a distinct and demanding frontier for large language models (LLMs), offering insights beyond general mathematical problem-solving. Progress in this area is hampered by existing… See the full description on the dataset page: https://huggingface.co/datasets/AI4Math/IneqMath.AIDA
Dataset Card for AIDABench
Links
Paper (arXiv)
GitHub Repository
Dataset Summary
AIDABench is a benchmark for evaluating AI systems on end-to-end data analytics over real-world documents. It contains 600+ diverse analytical tasks grounded in realistic scenarios and spans heterogeneous data sources such as spreadsheets, databases, financial reports, and operational records. Tasks are designed to be challenging, often requiring multi-step reasoning… See the full description on the dataset page: https://huggingface.co/datasets/MichaelYang-lyx/AIDA.FactCheck
Dataset Card for FactCheck
📝 Dataset Summary
FactCheck is an benchmark for evaluating LLMs on knowledge graph fact verification. It combines structured facts from YAGO, DBpedia, and FactBench with web-extracted evidence including questions, summaries, full text, and metadata. The dataset contains examples designed for sentence-level fact-checking and QA tasks.
📚 Supported Tasks
Question Answering: Answer fact-checking questions derived from KG triples.… See the full description on the dataset page: https://huggingface.co/datasets/FactCheck-AI/FactCheck.tw-drug-labels-vision
Dataset Card for tw-drug-labels-vision
💊 tw-drug-labels-vision 是一份涵蓋臺灣食品藥物管理署（TFDA）核發之 44,663 筆藥品仿單／外盒 的繁體中文多模態資料集。每一筆紀錄同時包含 PDF 全部頁面的渲染圖（WebP 多頁）以及一份依統一 17 欄 JSON Schema 抽取自原始藥品標示文件的結構化資料，可直接用於語言模型微調、視覺語言模型訓練、文件問答、藥品知識檢索、繁體中文醫藥 NLP 任務之素材。
Dataset Details
Dataset Description
本資料集源自臺灣 TFDA 公開的藥品許可證查詢系統。每筆紀錄對應一份藥品文件（仿單或外盒），原始為 PDF 圖檔形式。處理流程分為三階段：
下載：依據 20251222政府開放資料集_仿單與藥品外盒_66032.xlsx 中的 PDF URL，下載原始檔。
頁面渲染：將 PDF 各頁渲染為 WebP 圖檔，封裝在 images 欄位中。
OCR +… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/tw-drug-labels-vision.Kairos-Multimodal-Reasoning
A dataset for training models in multimodal reasoning tasks
Usage
from datasets import load_dataset
ds = load_dataset("Aquiles-ai/Kairos-Multimodal-Reasoning")
print(ds.features)
print(ds["train"]["source"])
Preview of dataset examples
We've built a playground so you can see some of the examples included in the dataset.
Link: https://kairos-example.vercel.app/
Dataset used in the blog post: Kairos: Building a Multimodal Model with LFM2.5 and… See the full description on the dataset page: https://huggingface.co/datasets/Aquiles-ai/Kairos-Multimodal-Reasoning.perceptual-constancy
Perceptual Constancy
Perceptual Constancy is a multimodal benchmark designed to evaluate high-level perceptual invariance in large vision-language models (VLMs). It probes a model’s understanding of physical and geometric stability under varying sensory appearances. This dataset is part of the Grow AI Like a Child benchmark initiative.
🧠 Dataset Overview
The Perceptual Constancy dataset focuses on appearance-invariant reasoning using both static images and short… See the full description on the dataset page: https://huggingface.co/datasets/grow-ai-like-a-child/perceptual-constancy.MuSLR
🧩 MuSLR: Multimodal Symbolic Logical Reasoning Benchmark
Project page: "Multimodal Symbolic Logical Reasoning".
Paper Link: https://arxiv.org/abs/2509.25851
Multimodal symbolic logical reasoning, which aims to deduce new facts from multimodal input via formal logic, is critical in high-stakes applications such as autonomous driving and medical diagnosis, where rigorous, deterministic reasoning helps prevent serious consequences.
To evaluate such capabilities of current… See the full description on the dataset page: https://huggingface.co/datasets/Aiden0526/MuSLR.AIME25-CoT-CN
Sci-Bench-AIME25'
This repo is a branch of Sci Bench made by IPF team. Mainly include the AIME 25' solution with multi-modal CoT and diverse solving path.
Brief intro
💻 Overview
A brief template and final report will be posted in Isaac's Blog
And the markdown template can be found in data/I_2
❓ Why we do this?
The multi-lingual datasets are scarce, while the CoT of Math is even less, no matter whether the CoT or the solution contains pictures… See the full description on the dataset page: https://huggingface.co/datasets/IPF/AIME25-CoT-CN.forge
FORGE: Fine-grained Multimodal Evaluation for Manufacturing Scenarios
🌐 Website | 📑 Paper | 💻 Code | 🤗 Dataset
Quick Start
from datasets import load_dataset
ds = load_dataset("AI4Manufacturing/forge", "task1_three_view", split="train")
print(ds[0].keys())
ds[0]["test_image"] # PIL Image
Configs
Core… See the full description on the dataset page: https://huggingface.co/datasets/AI4Manufacturing/forge.HanDyVQA
HanDyVQA Dataset 👋
HanDyVQA (Hand-Object Dynamics Video Question Answering) Dataset is a new benchmark for evalutating abundant spatio-temporal dynamics, process, and effects contained in hand-object interactions. This dataset is built on top of Ego4D Dataset.
Get Started
0. Install LFS
If you haven’t already, install Git Large File Storage (LFS):
git lfs install
1. Clone Repository
git clone https://huggingface.co/datasets/aist-cvrt/HanDyVQA… See the full description on the dataset page: https://huggingface.co/datasets/aist-cvrt/HanDyVQA.aec-bench
AEC-Bench: A Multimodal Dataset for Architecture, Engineering, and Construction
Section
What it covers
Overview
What the dataset contains
Task taxonomy
Scopes, task families, instance counts
Accessing the dataset
manifest.jsonl, prefetching files from URLs
License
Apache 2.0
Citation
BibTeX
Overview
AEC-Bench is a multimodal dataset of real-world Architecture, Engineering, and Construction (AEC) documents — construction drawings, floor… See the full description on the dataset page: https://huggingface.co/datasets/nomic-ai/aec-bench.ChemO
🧪 ChemO Dataset
📄 Paper: ChemLabs on ChemO: A Multi-Agent System for Multimodal Reasoning on IChO 2025
ChemO Version 1.1
Now with CDXML Files! 🎉
The ChemO dataset has been officially released after meticulous proofreading and preparation. This benchmark is built from the International Chemistry Olympiad (IChO) 2025 and represents a new frontier in automated chemical problem-solving.
🌟 Key Features
🏆 Olympic-Level Benchmark - Challenging problems… See the full description on the dataset page: https://huggingface.co/datasets/IDEA-AI4S/ChemO.MATRIX
MATRIX
MATRIX (Materials Analysis of Theory, Reasoning, and Images from eXperiments) is a multimodal dataset and benchmark released by Radical AI for evaluating reasoning capabilities in materials science across both textual and visual modalities.
MATRIX is designed primarily as a diagnostic and evaluation benchmark, rather than a large-scale training dataset. It targets structured scientific reasoning grounded in theory, experimental interpretation, and hypothesis generation.… See the full description on the dataset page: https://huggingface.co/datasets/radical-ai/MATRIX.M3CoTBenchArogyaBodha
ArogyaBodha
Multilingual multimodal medical VQA across English + 7 Indian languages. Long/per-language layout; ~40,857 rows; English rows separate; cid shared across languages.
Columns
Column
Type
Description
lid
Value('string')
Language-scoped id
language
Value('string')
Case language
Figure_A
Image(mode=None, decode=True)
Medical image
Figure_B
Image(mode=None, decode=True)
Medical image
Figure_C
Image(mode=None, decode=True)
Medical image… See the full description on the dataset page: https://huggingface.co/datasets/iit-patna-cse-ai/ArogyaBodha.Myanmar-Tuberculosis-Guidelines-Instructions
Myanmar Tuberculosis Guidelines Instructions
A bilingual instructional dataset built to support Myanmar's ongoing fight against tuberculosis — turning life-saving guidelines into a usable resource for healthcare workers, educators, and AI researchers working with low-resource languages.
Authors: Min Si Thu, Khin Myat Noe
Abstract
Tuberculosis is still one of Myanmar's biggest public health problems. Part of the difficulty is that good, standardized TB education… See the full description on the dataset page: https://huggingface.co/datasets/jojo-ai-mst/Myanmar-Tuberculosis-Guidelines-Instructions.Formosa-Vision
Dataset Card for Formosa-Vision
Formosa Vision 是一份以台灣在地文化為核心的開源視覺語言資料集，從國家文化記憶庫 2.0中精選兩千餘張資料，文字描述採用 OGDL 1.0 授權、及圖片為 CC By SA（及更開放的授權條款）授權的影像，內容涵蓋景點、建築、生活場景與歷史脈絡。資料集以模型生成與人工審核並行的方式建立，透過視覺語言模型產生影像對話，再由參與者逐一檢查與修訂，確保描述的正確性、文化脈絡的一致性與語句的自然性。專案由 Twinkle AI 社群發起，結合社群協作與開放文化精神，期待成為訓練繁體中文視覺語言模型的重要基礎，幫助研究者與開發者打造能真正理解台灣文化細節的 VLM 模型。
Dataset Details
Dataset Description
Formosa Vision（又稱 台灣視覺資料集）是一個以台灣在地視覺文化為核心、集結社群力量共創的開源資料集。這項專案源自近年視覺語言模型（Vision Language Model… See the full description on the dataset page: https://huggingface.co/datasets/twinkle-ai/Formosa-Vision.ArogyaBodha_ActCri
ArogyaBodha_ActCri
Parallel (wide) variant for the actor-critic framework; each row pairs a local-language case with its English translation. Splits: train ~32,250 / test 3,500 (symmetric 500-cid holdout).
Columns
Column
Type
Description
lid
Value('string')
Language-scoped id
language
Value('string')
Case language
Figure_A
Image(mode=None, decode=True)
Medical image
Figure_B
Image(mode=None, decode=True)
Medical image
Figure_C
Image(mode=None… See the full description on the dataset page: https://huggingface.co/datasets/iit-patna-cse-ai/ArogyaBodha_ActCri.MMPB
🌐 Project & Link
Official project page: https://aidaslab.github.io/MMPB/
Hugging Face dataset: 🤗 stackadd/MMPB
Paper (arXiv): “MMPB: It’s Time for Multi-Modal Personalization”
📘 Dataset Card for MMPB: Multi-Modal Personalization Benchmark
MMPB is the first large-scale benchmark designed to evaluate Vision-Language Models (VLMs) on personalization — their ability to understand, remember, and adapt to user-specific multimodal information.
It contains 10,000… See the full description on the dataset page: https://huggingface.co/datasets/snu-aidas/MMPB.AIME25-CoT-CN
Sci-Bench-AIME25'
This repo is a branch of Sci Bench made by IPF team-SnailAILab. Mainly include the AIME 25' solution with multi-modal CoT and diverse solving path.
📚 Cite
If you use the Sci-Bench-AIME25 (IPF/AIME25-CoT-CN) dataset in your research, please cite:
@dataset{zhang2025scibench_aime25,
title = {{Sci-Bench-AIME25}: A Multi-Modal Chain-of-Thought Dataset for Advanced Tool-Intergrated Mathematical Reasoning},
author = {Zhang, Haoxiang and Wang, Siyuan… See the full description on the dataset page: https://huggingface.co/datasets/SnailAILab/AIME25-CoT-CN.HCTQA
HCT-QA: Human-Centric Tables Question Answering
HCT-QA is a benchmark dataset designed to evaluate large language models (LLMs) on question answering over complex, human-centric tables (HCTs). These tables often appear in documents such as research papers, reports, and webpages and present significant challenges for traditional table QA due to their non-standard layouts and compositional structure.
The dataset includes:
2,188 real-world tables with 9,835 human-annotated QA… See the full description on the dataset page: https://huggingface.co/datasets/qcri-ai/HCTQA.Sujet-Finance-QA-Vision-100k
Dataset Description 📊🔍
The Sujet-Finance-QA-Vision-100k is a comprehensive dataset containing over 100,000 question-answer pairs derived from more than 9,800 financial document images. This dataset is designed to support research and development in the field of financial document analysis and visual question answering.
Key Features:
🖼️ 9,801 unique financial document images
❓ 107,050 question-answer pairs
🇬🇧 English language
📄 Diverse financial document types… See the full description on the dataset page: https://huggingface.co/datasets/sujet-ai/Sujet-Finance-QA-Vision-100k.Viet-qvq-r1
Description
This dataset is a Vietnamese translation of the ahmedheakl/qvq-r1, intended for training and evaluating multimodal Vision–Language Models (VLMs) on visual reasoning tasks involving document-style images such as receipts, forms, invoices.,
Each example includes:
An input image containing text (typically scanned documents),
A conversation simulating a user question and an assistant’s step-by-step reasoning leading to the answer,
A Vietnamese version of the full… See the full description on the dataset page: https://huggingface.co/datasets/5CD-AI/Viet-qvq-r1.three-mountain-scaling
ThreeMountain_Scaling
Segment
Meaning
GO
Geometric Object — indicates the object type used (e.g., GO for geometric, RO for real objects).
L / Arc
Object Arrangement — defines how objects are arranged spatially. L means L-shape arrangement; Arc means objects are placed in an arc.
RC
Random Character Position — RC = True: character position is randomized.
FC
Fixed Character Position — FC = True: character stays fixed.
RS
Random Scale — RS = True: objects are… See the full description on the dataset page: https://huggingface.co/datasets/grow-ai-like-a-child/three-mountain-scaling.mechanical-reasoning
Mechanical Reasoning
Mechanical Reasoning is a multimodal benchmark designed to evaluate mechanical and physical reasoning in large vision-language models (VLMs). This dataset is introduced in the paper:
Probing Mechanical Reasoning in Large Vision Language ModelsarXiv:2410.00318
It is part of the Grow AI Like a Child benchmark initiative, which explores structured, curriculum-aligned evaluation for embodied intelligence.
🧠 Dataset Overview
This dataset focuses on… See the full description on the dataset page: https://huggingface.co/datasets/grow-ai-like-a-child/mechanical-reasoning.Vietnamese-ShareGPT4Video-ShareGPT4Video-gg-translated