datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
synthesized_datasetResearchClawBench
ResearchClawBench
Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery
Quick Start | Submit Tasks | How It Works | Domains | Leaderboard | Add Your Agent
ResearchClawBench is a benchmark that measures whether AI coding agents can independently conduct scientific research — from reading raw data to producing publication-quality reports — and then rigorously evaluates the results against real human-authored papers.… See the full description on the dataset page: https://huggingface.co/datasets/InternScience/ResearchClawBench.yayi2_pretrain_data
介绍/Introduction
本数据集源自雅意训练语料，我们精选了约100B数据，数据大小约为500GB。我们期望通过雅意预训练数据的开源推动中文预训练大模型开源社区的发展，并积极为此贡献力量。通过开源，我们与每一位合作伙伴共同构建雅意大模型生态。
We opensource the pre-trained dataset in this release, it should contain more than 100B tokens depending on the tokenizer you use, requiring more than 500GB of local storage. By open-sourcing the pre-trained dataset, we aim to contribute to the development of the Chinese pre-trained large language model open-source community. Through open-source, we aspire to… See the full description on the dataset page: https://huggingface.co/datasets/wenge-research/yayi2_pretrain_data.benchmark-research
Benchmark Research
面向金融 Deep Research Agent 的评测数据集（仅数据）。
代码、builder、prompt 模板与评测脚本在独立代码仓库维护；本 Hub 仓库只发布标准化 ready seeds，支持按子集按需加载。
快速开始
from datasets import load_dataset
# 按题型子集加载（推荐）
a1 = load_dataset("sselaine27/benchmark-research", "a1")
c = load_dataset("sselaine27/benchmark-research", "c")
d = load_dataset("sselaine27/benchmark-research", "d")
print(a1["train"][0]["task_id"])
print(a1["train"][0]["prompt"][:200])
# 按时间层筛选
t2 = a1["train"].filter(lambda x:… See the full description on the dataset page: https://huggingface.co/datasets/sselaine27/benchmark-research.newswire
Dataset Card for NewsWire
Dataset Summary
NewsWire contains 2.7 million unique public domain U.S. news wire articles, written between 1878 and 1977. Locations in these articles are georeferenced, topics are tagged using customized neural topic classification, named entities are recognized, and individuals are disambiguated to Wikipedia using a novel entity disambiguation model.
Languages
English (en)
Dataset Structure
Each year in the dataset is… See the full description on the dataset page: https://huggingface.co/datasets/dell-research-harvard/newswire.bankertoolbench
BankerToolBench
BankerToolBench is a benchmark of 100 end-to-end investment banking tasks for
evaluating AI agents. Each task mirrors real junior-banker work — building
financial models, preparing pitch decks, writing memos — and produces multi-file
deliverables (Excel, PowerPoint, Word) that are scored against expert-authored
rubrics.
The benchmark was developed with 502 investment bankers from firms including
Goldman Sachs, JPMorgan, Evercore, and others. Human completion time… See the full description on the dataset page: https://huggingface.co/datasets/handshake-ai-research/bankertoolbench.data-product-benchmark
DPDisc Dataset
Paper | Code
Dataset Description
This dataset provides a benchmark for automatic data product creation. The task is framed as follows: given a natural language data product request and a corpus of text and tables, the objective is to identify the relevant tables and text documents that should be included in the resulting data product which would useful to the given data product request. The benchmark brings together three variants: HybridQA, TAT-QA, and… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/data-product-benchmark.lca-results
Long Code Arena (raw results)
These are the raw results from the Long Code Arena benchmark suite, as well as the corresponding model predictions.
Please use the subset dropdown menu to select the necessary data relating to our six benchmarks:
🤗 Library-based code generation
🤗 CI builds repair
🤗 Project-level code completion
🤗 Commit message generation🤗 Bug localization
🤗 Module summarization
Web-Bench
Web-Bench
English | 中文 README
📖 Overview
Web-Bench is a benchmark designed to evaluate the performance of LLMs in actual Web development. Web-Bench contains 50 projects, each consisting of 20 tasks with sequential dependencies. The tasks implement project features in sequence, simulating real-world human development workflows. When designing Web-Bench, we aim to cover the foundational elements of Web development: Web Standards and Web Frameworks. Given the scale and… See the full description on the dataset page: https://huggingface.co/datasets/bytedance-research/Web-Bench.lgg-mri-segmentation-research
LGG Brain MRI Segmentation with Genomic Clusters
This repository provides a Patient-Centric version of the Lower-Grade Glioma (LGG) Segmentation dataset. While other versions of this data exist, they often treat slices as independent images. This version preserves the 3D patient volume and integrates all genomic/clinical labels directly into a multimodal-ready format.
🌟 Why This Version?
Developed for Multimodal AI Research, this dataset addresses several limitations… See the full description on the dataset page: https://huggingface.co/datasets/Ehsan-rmz/lgg-mri-segmentation-research.gero-research-evidence-2026-09
GERO research evidence — 118 publications
This dataset contains 118 distinct report, case-study, experiment, preprint and research-map records, with individual Markdown pages. All previous 117 corpus rows, including the Collatz map, are preserved byte for byte. The newest addition is the FinancePy floating-lookback intermediate-overflow audit, with verified developer issue272 and explicit limitations. Report counts are not independent-defect counts.
Latest numerical… See the full description on the dataset page: https://huggingface.co/datasets/XamitK/gero-research-evidence-2026-09.excelforum-nemo-gymresearchscope-papers
ResearchScope Papers
Open CS research paper dataset maintained by ResearchScope.
Updated automatically via GitHub Actions.
Quick start
from datasets import load_dataset
ds = load_dataset("kishormorol/researchscope-papers", "papers", split="train")
print(ds[0])
See Usage below for per-source splits, instruction-tuning, and the per-section fine-tuning data.
Stats
33,467 papers (raw metadata) — 8,467 arXiv · 20,000 conference · 5,000 journal
166,887… See the full description on the dataset page: https://huggingface.co/datasets/kishormorol/researchscope-papers.AssetOpsBench
AssetOpsBench
AssetOpsBench is a specialized benchmark designed for evaluating Large Language Models (LLMs) and Multi-Agent systems in industrial operations. It focuses on the intersection of sensor data interpretation, maintenance logic, and Prognostics and Health Management (PHM).
The benchmark enables researchers to test how effectively AI agents can manage complex industrial assets, such as compressors and hydraulic pumps, by applying rule-based logic and diagnostic… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/AssetOpsBench.researchrubricshttps://www.arxiv.org/abs/2511.07685
commit-msg-edits
✍️ Commit Message Edits Dataset
This dataset is a collection of expert-labeled commit message edits contributed via Commit Message Editing app presented in Towards Realistic Evaluation of Commit Message Generation by Matching Online and Offline Settings.
Labelers were presented with GPT-4 generated messages for 15 commits from CMG benchmark from Long Code Arena and asked to manually edit them to be of good enough quality to submit to VCS.
You can check Manual tab in our… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/commit-msg-edits.ResearchMath-14k
ResearchMath-14k
ResearchMath-14k is a collection of 14,056 research-level mathematical problem records extracted from papers, open-problem lists, workshop sheets, and related academic sources. Each record contains the original extracted question, a rewritten self-contained problem statement, taxonomy labels, and open-status metadata.
Paper: ResearchMath-14K: Scaling Research-Level Mathematics via Agents
Load
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/amphora/ResearchMath-14k.headlines-semantic-similarity
Dataset Card for HEADLINES
Dataset Summary
HEADLINES is a massive English-language semantic similarity dataset, containing 396,001,930 pairs of different headlines for the same newspaper article, taken from historical U.S. newspapers, covering the period 1920-1989.
Languages
The text in the dataset is in English.
Dataset Structure
Each year in the dataset is divided into a distinct file (eg. 1952_headlines.json), giving a total of 70 files.
The… See the full description on the dataset page: https://huggingface.co/datasets/dell-research-harvard/headlines-semantic-similarity.llama-9b-bulk-npzEnvBench
🌱⚙️ EnvBench
This repository contains data associated with EnvBench benchmark from EnvBench: A Benchmark for Automated Environment Setup.
It contains:
statistics about repositories from GitHub Search under ghs/data folder;
several data splits under splits folder;
Git repositories under repos folder;
READMEs under readmes folder (available as readmes config);
GitHub Actions workflows under workflows folder (available as workflows config);
list of files in the repositories… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/EnvBench.DREAM-1K
DREAM-1K
DREAM-1K (Description
with Rich Events, Actions, and Motions) is a challenging video description benchmark. It contains a collection of 1,000 short (around 10 seconds) video clips with diverse complexities from five different origins: live-action movies, animated movies, stock videos, long YouTube videos, and TikTok-style short videos. We provide a fine-grained manual annotation for each video.
Bellow is the dataset statistics:
FoR-T2I
Can Text-to-Image Models Draw from the Right Frame of Reference?
Left is not always image-left. If a person is facing the viewer, their left hand appears on the right side of the image. A text-to-image model can therefore render a plausible scene with the requested objects while still drawing the spatial relation from the wrong perspective.
FoR-T2I is the official benchmark release for Can Text-to-Image Models Draw from the Right Frame of Reference?. It… See the full description on the dataset page: https://huggingface.co/datasets/ernie-research/FoR-T2I.Our1-2b-DatasetResearch-Intent-Judge
Research Intent — LLM-as-Judge
▶️ Watch the Video
LLM-as-Judge annotations for research paper intent classification, collected
through the Echo-DSRN collaborative platform during the OpenAIRE AI Hackathon 2026.
The dataset has one split per judge model (Gemma_4_E4B_it_GGUF,
Qwen3.6_35B_A3B_GGUF, Bonsai_8B_gguf, ...) plus a human_annotations
split with curator annotations. Split names use underscores in place of the
dashes in model names (HF does not allow dashes in split… See the full description on the dataset page: https://huggingface.co/datasets/ethicalabs/Research-Intent-Judge.noteflow-research-pilots
Keep the failed attempts. Check the artifact.
Versioned public development evidence from Robot Reel × Skills Anywhere × EvalArc, recorded 14 September 2026 on an NVIDIA L40S, with separate scripted Harbor controls on CPU and separate GPU context-control and agent-requested MCP handoff cohorts recorded 19 September 2026. This is an inspectable engineering casebook, not a held-out benchmark or training corpus with established efficacy.
Configuration
Actual experiment
What… See the full description on the dataset page: https://huggingface.co/datasets/glayguo/noteflow-research-pilots.SpatialGen-Testset
SpatialGen Testset
This repository contains the test set for SPATIALGEN: Layout-guided 3D Indoor Scene Generation, a novel multi-view multi-modal diffusion model for generating realistic and semantically consistent 3D indoor scenes.
Project page | Paper | Code
We provide a test set of 48 preprocessed point clouds and their corresponding GT layouts, multi-view images are cropped from the high-resolution panoramic images.
Folder Structure
Outlines of the dataset files:… See the full description on the dataset page: https://huggingface.co/datasets/manycore-research/SpatialGen-Testset.Split-IFEval
Split IFEval
This dataset modifies the Instruction-Following Eval (IFEval) benchmark to split apart the task from the syntactic instructions in addition to fixing errors in the original dataset.
It enables the use of research methods like attention steering that require access to the instruction text.
To load the dataset, run:
from datasets import load_dataset
split_ifeval = load_dataset("ibm-research/Split-IFEval")
Dataset Structure
Each entry in the dataset… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/Split-IFEval.FailureSensorIQ
FailureSensorIQ Dataset
FailureSensorIQ is a Multi-Choice QA (MCQA) dataset that explores the relationships between sensors and failure modes for 10 industrial assets.
|Github | 🏆Leaderboard | 📖Paper |
Dataset Summary
FailureSensorIQ is a Multi-Choice QA (MCQA) dataset that explores the relationships between sensors and failure modes for 10 industrial assets. By only leveraging the information found in ISO documents, we developed a data generation pipeline that… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/FailureSensorIQ.BioHopR
BioHopR
Paper |
Description
We introduce BioHopR, a novel benchmark designed to evaluate multi-hop, multi-answer reasoning in structured biomedical knowledge graphs.Built from the comprehensive PrimeKG, BioHopR includes 1-hop and 2-hop reasoning tasks that reflect real-world biomedical complexities.
Prompt
We used the below to get the response of the open source LLMs.
def generate_single(model, tokenizer, question):
q="You are an expert biomedical… See the full description on the dataset page: https://huggingface.co/datasets/knowlab-research/BioHopR.fable5-traces-sft
Fable 5 Traces — Unified SFT / Self-Distillation Dataset
A cleaned, unified, PII-scrubbed corpus of Claude Fable 5 agent traces in
OpenAI-style chat format, plus a working on-policy self-distillation (SDFT)
training scaffold.
Composition
Source
Conversations
Claude Code raw agentic sessions
18
CoT distillation records
4,665
Unique conversations (post-dedup)
4,683
Split deterministically by content hash: train 4,442 / validation 241.
The raw… See the full description on the dataset page: https://huggingface.co/datasets/Swarm-AI-Research/fable5-traces-sft.