datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
jat-dataset
JAT Dataset
Dataset Description
The Jack of All Trades (JAT) dataset combines a wide range of individual datasets. It includes expert demonstrations by expert RL agents, image and caption pairs, textual data and more. The JAT dataset is part of the JAT project, which aims to build a multimodal generalist agent.
Paper: https://huggingface.co/papers/2402.09844
Usage
>>> from datasets import load_dataset
>>> dataset =… See the full description on the dataset page: https://huggingface.co/datasets/jat-project/jat-dataset.EarthDial-Dataset
🌍 EarthDial-Dataset
The EarthDial-Dataset is a curated collection of evaluation-only datasets focused on remote sensing and Earth observation downstream tasks. It is designed to benchmark vision-language models (VLMs) and multimodal reasoning systems on real-world scenarios involving satellite and aerial imagery.
📚 Key Features
Evaluation-focused: All datasets are for inference/testing only — no train/val splits.
Diverse Tasks:
Classification
Object Detection
Change… See the full description on the dataset page: https://huggingface.co/datasets/akshaydudhane/EarthDial-Dataset.EASI-Leaderboard-Data
EASI Leaderboard Data
A consolidated dataset for the EASI Leaderboard, containing the evaluation data (inputs/prompts) actually used on the leaderboard across spatial reasoning benchmarks for VLMs.
Looking for the Spatial Intelligence leaderboard?https://huggingface.co/spaces/lmms-lab-si/EASI-Leaderboard
🔎 Dataset Summary
Question types: MCQ (multiple choice) and NA (numeric answer).
File format: TSV only.
Usage: These TSVs are directly consumable by the EASI… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab-si/EASI-Leaderboard-Data.Curr-ReFT-data
Curr-ReFT-data
[📂 GitHub][📝 Paper]
[🤗 HF Dataset] [🤗 HF-Model: Curr-ReFT-3B]
[🤗 HF-Model: Curr-ReFT-7B]
Dataset Overview
Curr-ReFT-data contains training data for both stages of the Curr-ReFT methodology. The proposed Curr-ReFT post-training paradigm consists of two consecutive training stages: 1. Curriculum Reinforcement Learning: Gradually increasing task difficulty through reward mechanisms that match task complexity. 2. Rejected Sample based… See the full description on the dataset page: https://huggingface.co/datasets/ZTE-AIM/Curr-ReFT-data.MM-UAVBench
MM-UAVBench
A comprehensive multimodal benchmark designed to evaluate the perception, cognition, and planning abilities of Multimodal Large Language Models (MLLMs) in low-altitude UAV scenarios.
📚 Dataset Overview
MM-UAVBench focuses on assessing MLLMs' performance in UAV-specific low-altitude scenarios, with three core characteristics:
Key Features
Comprehensive Task Design
19 tasks across 3 capability dimensions (perception/cognition/planning)… See the full description on the dataset page: https://huggingface.co/datasets/daisq/MM-UAVBench.LLAVA-LibMoE
Download Instructions
This repository provides the full LLAVA-LibMoE dataset, including LLaVA-665K and OneVision-1M2 image sources, organized into the following required directory tree:
libmoe/
└── data/
├── image_onevision/
├── coco/
│ └── train2017/
├── gqa/
│ └── images/
├── ocr_vqa/
│ └── images/
├── textvqa/
│ └── train_images/
└── vg/
├── VG_100K/
└──… See the full description on the dataset page: https://huggingface.co/datasets/DavidNguyen/LLAVA-LibMoE.propagator-multimodal-pretraining-data
Propagator Multimodal Pretraining Data
This public dataset contains tokenized multimodal pretraining data prepared for the Propagator model family. It combines language, image-grounded, and speech/audio-token examples into a single training format.
This is not a raw text or image browsing dataset. The examples have already been converted into compact binary token frames for model training, with a manifest that records the source groups and file layout.
Source Code… See the full description on the dataset page: https://huggingface.co/datasets/ken-sungmin/propagator-multimodal-pretraining-data.TransCity-VLM-dataset
TransCity-VLM Dataset
The TransCity-VLM Dataset provides multimodal smart-city data for traffic, energy, mobility, grid operation, urban context understanding, and map-grounded question answering. It supports the training and evaluation of vision-language models for urban prediction, decision support, conversational QA, and reasoning tasks.
The training data are available at this Hugging Face dataset repository.
Dataset Summary
Split
Rows / Files
test JSONL… See the full description on the dataset page: https://huggingface.co/datasets/TransCity-VLM/TransCity-VLM-dataset.InternVL-Chat-V1-2-SFT-Data
Data Card for InternVL-Chat-V1-2-SFT-Data
Overview
Inspired by LLaVA-NeXT, we adopted a data-efficient SFT strategy to train InternVL-Chat-V1-2, utilizing approximately 1.2M of visual instruction tuning samples in total, all of which are fully open-source. In a macro sense, we build upon ShareGPT-4V and additionally integrate LLaVA-ZH, DVQA, ChartQA, AI2D, DocVQA, GeoQA+, and SynthDoG-EN. Most of the data remains consistent with LLaVA-NeXT.
Citation
If you use… See the full description on the dataset page: https://huggingface.co/datasets/OpenGVLab/InternVL-Chat-V1-2-SFT-Data.Uni-MuMER-Data
Uni-MuMER: Unified Multi-Task Vision-Language Model Fine-Tuning for Handwritten Mathematical Expression Recognition
This repository provides datasets and fine-tuned models related to Uni-MuMER, a unified multi-task fine-tuning framework designed to enhance Vision-Language Models (VLMs) for Handwritten Mathematical Expression Recognition (HMER).
Uni-MuMER integrates domain-specific knowledge directly into generalist VLM architectures without modification, employing data-driven… See the full description on the dataset page: https://huggingface.co/datasets/phxember/Uni-MuMER-Data.ARFBench
Dataset Card for ARFBench (Anomaly Reasoning Framework Benchmark)
Links
Paper
Toto-1.0-QA-Experimental Model
Leaderboard
Github
Dataset Summary
ARFBench (Anomaly Reasoning Framework Benchmark) is a multimodal time-series reasoning benchmark consisting of 750 question-answer (QA) pairs composed from
real-world incident data collected at Datadog, a leading observability platform.
Note: the metrics comprising ARFBench were generated from internal monitoring and… See the full description on the dataset page: https://huggingface.co/datasets/Datadog/ARFBench.t5gemma2-indonesia-instruct-v1
T5Gemma-2 Indonesian Instruct — Mono-Repo
Satu repositori dataset HF untuk seluruh data pelatihan T5-Gemma-2 bahasa Indonesia.
Diorganisasi per fungsi (fondasi → spesifik → preferensi) dengan folder/subfolder,
setiap config = folder dan berisi split train + validation (80:20) di level percakapan.
Struktur (by fungsi)
t5gemma2-indonesia-instruct-v1/
├── README.md
├── manifest.json
├── chat_idx_map.json
├── foundation/ ← FASE 1 · fondasi Bahasa… See the full description on the dataset page: https://huggingface.co/datasets/daruokta/t5gemma2-indonesia-instruct-v1.MMMU
MMMU (A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI)
🌐 Homepage | 🏆 Leaderboard | 🤗 Dataset | 🤗 Paper | 📖 arXiv | GitHub
🔔News
🛠️[2026-04-21]: Fixed option issue in test_Psychology_15.
‼️[2026-02-12]: We have released the answers for the test set! You can now evaluate your models on the test set locally! 🎉
🛠️[2024-05-30]: Fixed duplicate option issues in Materials dataset items (validation_Materials_25;… See the full description on the dataset page: https://huggingface.co/datasets/Daikaku/MMMU.STRIDE-QA-Dataset-Mini
STRIDE-QA-Dataset-Mini
STRIDE-QA is a large-scale visual question answering (VQA) dataset for physically grounded spatiotemporal reasoning in autonomous driving. Constructed from 100 hours of multi-sensor driving data in Tokyo, it offers 16 M QA pairs over 270 K frames with dense annotations including 3D bounding boxes, segmentation masks, and multi-object tracks.
⚠️ Note: STRIDE-QA-Dataset-Mini is provided as a preliminary version and does not fully match the format of the… See the full description on the dataset page: https://huggingface.co/datasets/turing-motors/STRIDE-QA-Dataset-Mini.data-estate
Quick start — get everything in one command
pip install -U huggingface_hub
hf download EMTIAZZ/data-estate --repo-type dataset --local-dir ./data-estate
This downloads everything (all tables, PDFs, scanned images, emails, and transcripts) into
./data-estate. See How to download for more ways.
Hints
These are hints, not answers. They point out what to look for in each kind of data and what
to think about. Picking the tools and building the pipeline is your job.… See the full description on the dataset page: https://huggingface.co/datasets/EMTIAZZ/data-estate.azerbaijan-court-data
Azerbaijan Court System Dataset
The most comprehensive open dataset of Azerbaijan's judicial system — 1.64 million structured records and 1.54 million court decision PDFs (~160 GB) covering court decisions, active cases, scheduled hearings, court registries, judges, lawyers, and mediator organizations.
Built for AI engineers, legal tech startups, and researchers who need real-world legal data at scale.
Quick Start
Load with Hugging Face datasets
from datasets… See the full description on the dataset page: https://huggingface.co/datasets/ismatsamadov/azerbaijan-court-data.Flux_SD3_MJ_Dalle_Human_Coherence_Dataset
NOTE: A newer version of this dataset is available: Imagen3_Flux1.1_Flux1_SD3_MJ_Dalle_Human_Coherence_Dataset
Rapidata Image Generation Coherence Dataset
This Dataset is a 1/3 of a 2M+ human annotation dataset that was split into three modalities: Preference, Coherence, Text-to-Image Alignment.
Link to the Preference dataset: https://huggingface.co/datasets/Rapidata/700k_Human_Preference_Dataset_FLUX_SD3_MJ_DALLE3
Link to the Text-2-Image Alignment dataset:… See the full description on the dataset page: https://huggingface.co/datasets/Rapidata/Flux_SD3_MJ_Dalle_Human_Coherence_Dataset.CaReCoS
CaReCoS
A medical acoustic question-answering dataset for reasoning over mel spectrograms
of heart, lung, and cough sounds. Each record provides a clinical question, the
mel-spectrogram image of a recording, a ground-truth answer, and the
recording's clinical metadata.
The task is purely visual: a model receives the spectrogram image together with the
question and must reason over the spectrogram to produce the answer. The raw audio is
not used as model input - the original .wav… See the full description on the dataset page: https://huggingface.co/datasets/anonymous-submission-dataset-1/CaReCoS.Medical_Multimodal_Evaluation_Data
Evaluation Guide
This dataset is used to evaluate medical multimodal LLMs, as used in HuatuoGPT-Vision. It includes benchmarks such as VQA-RAD, SLAKE, PathVQA, PMC-VQA, OmniMedVQA, and MMMU-Medical-Tracks.
To get started:
Download the dataset and extract the images.zip file.
Find evaluation code on our GitHub: HuatuoGPT-Vision.
This open-source release aims to simplify the evaluation of medical multimodal capabilities in large models. Please cite the relevant benchmark… See the full description on the dataset page: https://huggingface.co/datasets/FreedomIntelligence/Medical_Multimodal_Evaluation_Data.data-agent-benchmarks
LongHorizon Full Data-Agent Benchmarks
Companion data artifacts for five complete evaluation tracks:
DataSciBench full55 / 167 metric entries
DABStep full450
DABStep-Research full100
DSBench Modeling full74
LongDS full68 / 2,225 turns
The companion GitHub repository contains processed manifests, evaluation code,
historical API ReAct baseline code, download/preparation tools, and the frozen
source lock. artifact_manifest.json records every uploaded object's size,
SHA-256… See the full description on the dataset page: https://huggingface.co/datasets/noel7Y/data-agent-benchmarks.letterboxd-all-movie-data
Letterboxd Film Dataset
This dataset contains a comprehensive collection of 847,209 films from the Letterboxd platform, including movie information, user reviews, and ratings.
Dataset Summary
Total Films: 847,209
File Size: ~1.12 GB (1,120,572,122 bytes)
Format: JSONL (JSON Lines)
Language: Primarily English, with some multilingual content
Data Structure
Each line contains a JSON object with the following fields:
{
"url":… See the full description on the dataset page: https://huggingface.co/datasets/pkchwy/letterboxd-all-movie-data.norwegian-dyna-instruct
🧨 Norwegian dyna-instruct
Version
0.1.0 (changelog)
Languages
Norwegian Bokmål (nob), Norwegian Nynorsk (nno), and English (eng) translation input
License
Mixed open licenses; see the table below
Sources
Five datasets (source cards)
Dataset Description
Number of samples: 14.40K
Number of tokens (Llama 3): 6.27M
Average conversation length in tokens (min, max): 435.63 (4, 8.92K)
Average number of turns (min, max): 2.13 (2, 3)… See the full description on the dataset page: https://huggingface.co/datasets/danish-foundation-models/norwegian-dyna-instruct.n8n-Toolkit
🤖 n8n-Toolkit Dataset
A comprehensive fine-tuning dataset for n8n workflow automation, AI agents, SEO/marketing strategist, and business automation
📊 Quick Stats
📈 Metric
Count
Percentage
Total Examples
55,026
100%
Estimated Pages
550
-
With Images/Screenshots
54,03898.2%
With System Messages
~55,000
~100%
With Thinking Supervision
~30,000+
~55%
With Task Labels
55,026
100%
🎯 What Is This Dataset?
This is a… See the full description on the dataset page: https://huggingface.co/datasets/DavidrPatton/n8n-Toolkit.SenseNova-MARS-DataThis dataset provides the Wikipedia corpus and fully processed training and evaluation data that can be used out of the box for SenseNova-MARS. To ensure the training and evaluation scripts function correctly, these resources must be downloaded and placed into your code directory following the structure below.
📂 Dataset Contents
The dataset consists of the following processed components:
Wikipedia Corpus (wiki_20250901/): A pre-processed knowledge base containing Faiss index… See the full description on the dataset page: https://huggingface.co/datasets/sensenova/SenseNova-MARS-Data.ArtiFact
ArtiFact
ArtiFact is a large-scale multimodal benchmark of museum artwork records with aligned images and structured metadata. It is designed for evaluating metadata extraction, error detection, semantic querying, and multimodal reasoning over cultural-heritage collections.
The dataset combines records from the Rijksmuseum, the Metropolitan Museum of Art (Met), and the Art Institute of Chicago (AIC), with normalized fields for artists, dates, materials, techniques, dimensions… See the full description on the dataset page: https://huggingface.co/datasets/deem-data/ArtiFact.MMFineReason-1.8M-Qwen3-VL-235B-Thinking
MMFineReason
Closing the Multimodal Reasoning Gap via Open Data-Centric Methods
Average score across mathematical reasoning and multimodal understanding benchmarks.
📖 Overview
MMFineReason is a large-scale, high-quality multimodal reasoning dataset comprising 1.8M samples and 5.1B solution tokens, featuring detailed reasoning annotations distilled from Qwen3-VL-235B-A22B-Thinking.
🎯 Key Highlights
1.8M High-Quality Samples with 5.1B Solution Tokens… See the full description on the dataset page: https://huggingface.co/datasets/dans25275/MMFineReason-1.8M-Qwen3-VL-235B-Thinking.Inst-It-Dataset
Inst-IT Dataset: An Instruction Tuning Dataset with Multi-level Fine-Grained Annotations
introduced in the paper Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning
🌐 Homepage | Code | 🤗 Paper | 📖 arXiv
Inst-IT Dataset Overview
We create a large-scale instruction tuning dataset, the Inst-it Dataset. To the best of our knowledge, this is the first dataset that provides fine-grained annotations centric on specific… See the full description on the dataset page: https://huggingface.co/datasets/Inst-IT/Inst-It-Dataset.ContPhy_Dataset
ContPhy Dataset Repository
ContPhy: Continuum Physical Concept Learning and Reasoning from Videos
Zhicheng Zheng*, Xin Yan*, Zhenfang Chen*, Jingzhou Wang, Qin Zhi Eddie Lim, Joshua B. Tenenbaum, and Chuang Gan (* denotes equal contributions)
ICML 2024Links | Project Page | Paper (Arxiv) | Codebase | Cite ContPhy
Structure
Mini Dataset
contphy_mini.zip: 20 videos each scenario with annotations
Full Dataset… See the full description on the dataset page: https://huggingface.co/datasets/zzcnewly/ContPhy_Dataset.vecombot-dataset
VECOM — Bộ dữ liệu thị trường Thương mại điện tử Việt Nam (VEComBot)
Bộ dữ liệu phụ lục cho đồ án tốt nghiệp VEComBot — hệ thống Đa tác tử (Multi-Agent
System) phân tích và tổng hợp thị trường Thương mại điện tử Việt Nam (VECOM). Đây là
kho tài liệu nguồn và corpus đã qua xử lý (figure-aware) được nạp vào PostgreSQL/pgvector
để phục vụ cả nhánh MAS lẫn nhánh baseline naive RAG.
Mục đích: dùng cho nghiên cứu học thuật và tái lập kết quả đồ án. Các báo cáo gốc là
ấn phẩm công… See the full description on the dataset page: https://huggingface.co/datasets/binhtran23/vecombot-dataset.PaperAudit_Dataset
PaperAudit Origin Data
This directory contains the original paper data downloaded and preprocessed for the PaperAudit project. The data includes papers from top-tier machine learning conferences with their parsed content, metadata, synthetic error annotations, and review information.
PaperAudit Dataset Overview
This repository is part of the full PaperAudit Dataset, which includes:
PaperAudit_Dataset/
├── PaperAudit_Origin_Data/ # Original paper data (raw + preprocessed)… See the full description on the dataset page: https://huggingface.co/datasets/mayiwen/PaperAudit_Dataset.