datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MathInstruct
🦣 MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning
MathInstruct is a meticulously curated instruction tuning dataset that is lightweight yet generalizable. MathInstruct is compiled from 13 math rationale datasets, six of which are newly curated by this work. It uniquely focuses on the hybrid use of chain-of-thought (CoT) and program-of-thought (PoT) rationales, and ensures extensive coverage of diverse mathematical fields.
Project Page:… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/MathInstruct.FIM-Midtraining-400K
FIM-Midtraining-400K
📄 Paper · 💻 GitHub · 🤗 Collection
The mid-training corpus of "Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models": 400K function-aware FIM samples (~2.6B tokens under the Qwen2.5-Coder tokenizer) drawn from 75,568 Python files across 968 permissively-licensed GitHub repositories, fully decontaminated against SWE-Bench.
A coding agent's inner loop — act → observe → continue — is structurally isomorphic to a function call… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/FIM-Midtraining-400K.pile-val-backupThis is a backup for the pile val dataset downloaded from here: https://the-eye.eu/public/AI/pile/val.jsonl.zst
Please respect the original license of the dataset.
RegexEval
Dataset Card for RegexEval
Re(gEx|DoS)Eval is a framework that includes a dataset of 762 regex descriptions (prompts) from real users, refined prompts with examples, and a robust set of tests.
Dataset Details
Dataset Sources
Repository: https://github.com/s2e-lab/RegexEval
Paper: https://s2e-lab.github.io/preprints/icse_nier24-preprint.pdf
Dataset Structure
dataset.jsonl: dataset file in jsonl format. Every line contains a JSON object with… See the full description on the dataset page: https://huggingface.co/datasets/s2e-lab/RegexEval.minWM-datawebfaq-retrievalWebFAQ Retrieval Dataset
Overview |
Details |
Structure |
Examples |
Considerations |
License |
Citation |
Contact |
Acknowledgement
Overview
The WebFAQ Retrieval Dataset is a carefully filtered and curated subset of the broader WebFAQ Q&A Dataset.It is purpose-built for Information Retrieval (IR) tasks, such as training and evaluating dense or sparse retrieval models in multiple languages.
Each of the… See the full description on the dataset page: https://huggingface.co/datasets/PaDaS-Lab/webfaq-retrieval.SuperMemory-VQA
SuperMemoryVQA
SuperMemory-VQA is an egocentric visual question answering benchmark for
evaluating long-horizon memory in augmented reality assistant settings. The
dataset is designed around practical questions a person might ask a wearable
memory assistant, such as where an object was left, what someone said earlier,
whether a planned step was completed, or what happened next in a longer event.
The benchmark contains 4,853 human-verified question-answer pairs grounded in
52.9… See the full description on the dataset page: https://huggingface.co/datasets/OSU-AIoT-MLSys-Lab/SuperMemory-VQA.MultiviewX_Labelsphysical-ai-bench-generation
Physical AI Bench - Generation
Paper | Code
Dataset Description
The PAI-Bench is a benchmark to measure the progress of world models quantitatively.
The predict task contains a list of 1044 samples of text prompts, conditioning images, and qa pairs, covering Physical AI target domains including autonomous vehicle (AV) driving, robotics, industry (smart space), physics, human, and common sense. All the questions are binary questions, and the answer is either Yes or No. Our… See the full description on the dataset page: https://huggingface.co/datasets/shi-labs/physical-ai-bench-generation.M-BEIR
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers (ECCV 2024)
🌐 Homepage | 🤗 Model(UniIR Checkpoints) | 🤗 Paper | 📖 arXiv | GitHub
How to download the M-BEIR Dataset
🔔News
🔥[2023-12-21]: Our M-BEIR Benchmark is now available for use.
Dataset Summary
M-BEIR, the Multimodal BEnchmark for Instructed Retrieval, is a comprehensive large-scale retrieval benchmark designed to train and evaluate unified multimodal retrieval… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/M-BEIR.HarmfulQAPaper | Github | Dataset| Model
📣📣📣: Do check our new multilingual dataset CatQA here used in Safety Vectors:📣📣📣
As a part of our research efforts toward making LLMs more safe for public use, we create HarmfulQA i.e. a ChatGPT-distilled dataset constructed using the Chain of Utterances (CoU) prompt. More details are in our paper Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
HarmfulQA serves as both-a new LLM safety benchmark and an alignment dataset… See the full description on the dataset page: https://huggingface.co/datasets/declare-lab/HarmfulQA.CoRECoRE: Controlled Retrieval Evaluation Dataset
Motivation |
Dataset Overview |
Dataset Construction |
Dataset Structure |
Qrels Format |
Evaluation |
Citation |
Links |
Contact
CoRE (Controlled Retrieval Evaluation) is a benchmark dataset designed for the rigorous evaluation of embedding compression techniques in information retrieval.
🔍 Motivation
Embedding compression is essential for scaling… See the full description on the dataset page: https://huggingface.co/datasets/PaDaS-Lab/CoRE.SABER
SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
SABER is the code release for the paper. It includes the benchmark tasks, sandbox runtime, judging pipeline, and baseline reproduction utilities used to evaluate operational safety in stateful project workspaces.
What is included
tasks/: benchmark task definitions and metadata
run_osbench.py, judge_osbench.py: historical inference and judging entry points
sandbox_shell.py… See the full description on the dataset page: https://huggingface.co/datasets/sssr-lab/SABER.webfaqWebFAQ Q&A Dataset
Overview |
Details |
Structure |
Examples |
Considerations |
License |
Citation |
Contact |
Acknowledgement
Overview
The WebFAQ Q&A Dataset is a broad-coverage corpus of 96 million natural question-answer (QA) pairs in 75 languages, gathered from FAQ pages on the web. It leverages structured schema.org FAQPage annotations, making it a unique resource for large-scale Question Answering… See the full description on the dataset page: https://huggingface.co/datasets/PaDaS-Lab/webfaq.sudoku-700kChiPBench-D
ChiPBench-D
ChiPBench:Benchmarking End-to-End Performance of AI-based Chip Placement Algorithms
Chip placement is a critical step in the Electronic Design Automation (EDA) workflow, which aims to arrange chip modules on the canvas to optimize the performance, power, and area (PPA) metrics of final designs.Recent advances show great potential of AI-based algorithms in chip placement.However, due to the lengthy EDA workflow, evaluations of these algorithms often focus on intermediate… See the full description on the dataset page: https://huggingface.co/datasets/MIRA-Lab/ChiPBench-D.gspc-human-labour-index
GSPC — labour components facts (Eurostat)
SWIFT census (live): https://councilof.ai/api/swift
XRPL reader (live): https://councilof.ai/api/xrpl
Live axis name: labour-components — MEASURED as two labour series (deterministic-facts, n=2). Not an index. No composite. C-2026-0826-05: do not restore MEASURED-INDEX-v0.1.
Legacy Hub slug kept for inbound links. Cite the live axis name. Empty cells that the GET does not fill stay empty.
Council of AI · CSOAI Ltd (GB, Companies House… See the full description on the dataset page: https://huggingface.co/datasets/csoai/gspc-human-labour-index.gspc-humanoid-labour-index
GSPC — humanoid labour index facts (Disclosure)
SWIFT census (live): https://councilof.ai/api/swift
XRPL reader (live): https://councilof.ai/api/xrpl
MEASURED financial/domain axis as disclosure facts on 8 frozen URLs (n=8). Not a model leaderboard. No accuracy, no fleet, no leader. Cells the live GET leaves empty stay empty — never invent hours/incidents.
Live status is the humanoid-labour-index row on GET https://councilof.ai/api/gspc. Not a certificate.
Council of AI ·… See the full description on the dataset page: https://huggingface.co/datasets/csoai/gspc-humanoid-labour-index.vlm-info-loss-results
VLM Grounding Evaluation Results
Grounding evaluation results for vision-language models on robotics manipulation datasets.
Part of the vlm-info-loss project studying
how VLM connectors transform visual representations.
Background
Our embedding-level analysis shows VLM connectors perform a compress-then-expand transformation:
they sharpen dominant-object representations while compressing secondary-object category identity.
All tested models converge to ~83%… See the full description on the dataset page: https://huggingface.co/datasets/MicroAGI-Labs/vlm-info-loss-results.QIT
QIT Humanize-Physic Formalizations and Proofs
QIT (Quantum Information Theory) is a blind benchmark for formalizing theorems in quantum information. It evaluates whether an AI agent can faithfully translate natural-language and TeX problem statements into Lean 4 theorems and then construct formal proofs checked by the Lean kernel. Its 40 tasks cover quantum channels and Choi representations, entropy and coding, mixed-unitary obstructions and symmetry, norm and fidelity tools… See the full description on the dataset page: https://huggingface.co/datasets/humanfia-lab/QIT.MusicBench
MusicBench Dataset
The MusicBench dataset is a music audio-text pair dataset that was designed for text-to-music generation purpose and released along with Mustango text-to-music model. MusicBench is based on the MusicCaps dataset, which it expands from 5,521 samples to 52,768 training and 400 test samples!
Dataset Details
MusicBench expands MusicCaps by:
Including music features of chords, beats, tempo, and key that are extracted from the audio.
Describing these music… See the full description on the dataset page: https://huggingface.co/datasets/amaai-lab/MusicBench.SudokuJustify
SudokuJustify
SudokuJustify is a benchmark for evaluating whether a model can produce stepwise, machine-checkable justifications for simple Sudoku reasoning. Each instance contains a 9x9 puzzle, its full solution, and a canonical symbolic trace using a restricted rule set:
naked_single
hidden_single
The benchmark is designed for evaluation and analysis rather than training. The restricted rule set keeps every claimed step easy to verify automatically.
Dataset Layout… See the full description on the dataset page: https://huggingface.co/datasets/anon-lab/SudokuJustify.PhysBench
PhysBench
🌐 Homepage | 🤗 Dataset | 📑 Paper | 💻 Code | 🔺 EvalAI
This repo contains evaluation code for the paper "PhysBench: Benchmarking and Enhancing VLMs for Physical World Understanding"
If you like our project, please give us a star ⭐ on GitHub for latest update.
Introduction
Understanding the physical world is a fundamental challenge in embodied AI, critical for enabling agents to perform complex tasks and operate safely in real-world environments. While… See the full description on the dataset page: https://huggingface.co/datasets/USC-PSI-Lab/PhysBench.CategoricalHarmfulQA
CatQA: A categorical harmful questions dataset
CatQA is used in LLM safety realignment research:
Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic (Paper, Code)
How to download
from datasets import load_dataset
dataset = load_dataset("declare-lab/CategoricalHarmfulQA")
What is CatQA?
To comprehensively evaluate the model across a wide range of harmful categories, we construct a new safety… See the full description on the dataset page: https://huggingface.co/datasets/declare-lab/CategoricalHarmfulQA.webfaq-bitextsWebFAQ Bilingual Datasets (Bitexts)
Overview |
Details |
Structure |
Examples |
Considerations |
License |
Citation |
Contact |
Acknowledgement
Overview
The WebFAQ Bilingual Datasets (a.k.a. Bitexts) are derived from the WebFAQ Q&A Dataset, but instead of monolingual question-answer (QA) pairs, each entry here contains aligned QA pairs in two different languages. These alignments are created via… See the full description on the dataset page: https://huggingface.co/datasets/PaDaS-Lab/webfaq-bitexts.lm-eval-results-princeton-nlp-Llama-3-Base-8B-SFT-RDPO-private
Dataset Card for Evaluation run of princeton-nlp/Llama-3-Base-8B-SFT-RDPO
Dataset automatically created during the evaluation run of model princeton-nlp/Llama-3-Base-8B-SFT-RDPO
The dataset is composed of 62 configuration(s), each one corresponding to one of the evaluated task.
The dataset has been created from 7 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the… See the full description on the dataset page: https://huggingface.co/datasets/nyu-dice-lab/lm-eval-results-princeton-nlp-Llama-3-Base-8B-SFT-RDPO-private.SWE-Next
SWE-Next: Scalable Real-World Software Engineering Tasks for Agents
SWE-Next Dataset
SWE-Next is an execution-grounded dataset of 2,308 self-verifying software engineering tasks mined from real merged GitHub pull requests. Starting from 3,971 seeded Python repositories and 102,582 executed candidate base/merged commit pairs, SWE-Next retains only instances where the merged commit produces a strict test improvement without regressions. The final release… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/SWE-Next.SWE-QA-Pro-Bench
SWE-QA-Pro Bench (A Repository-level QA Benchmark Built from Diverse Long-tail Repositories)
💻 GitHub | 📖 Paper | 🤗 SWE-QA-Pro
📢 News
🚀 [2026-5-19] The evaluation code is released on GitHub.
🔥 [2026-3-23] SWE-QA-Pro Bench is publicly released! The model and code will be released soon.
Introduction
SWE-QA-Pro Bench is a repository-level question answering dataset designed to evaluate whether models can perform grounded, agentic reasoning… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/SWE-QA-Pro-Bench.AIME25The AIME25 part 1 exam from the website.
lm-eval-results-AurelPx-Pegasus-7b-slerp-private
Dataset Card for Evaluation run of AurelPx/Pegasus-7b-slerp
Dataset automatically created during the evaluation run of model AurelPx/Pegasus-7b-slerp
The dataset is composed of 62 configuration(s), each one corresponding to one of the evaluated task.
The dataset has been created from 6 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
An… See the full description on the dataset page: https://huggingface.co/datasets/nyu-dice-lab/lm-eval-results-AurelPx-Pegasus-7b-slerp-private.