datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
adaptive-adversaries-data
Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
A 21-scenario multi-turn (15-round) adversarial red-teaming benchmark for LLM agents, in which both attacker and defender are independent LLM agents and attacks are regenerated per battle. Includes calibrated 3×3 attacker × defender matrix evaluation, full battle transcripts, attack-replay corpus, and traces from two open AgentBeats competitions.
Companion paper: Adaptive Adversaries: A Multi-Turn… See the full description on the dataset page: https://huggingface.co/datasets/neurips-adaptive-adversaries/adaptive-adversaries-data.neurips-2025-papers
NeurIPS 2025 Papers Dataset
This dataset contains all accepted papers from NeurIPS 2025, scraped from OpenReview.
Dataset Statistics
Overview
Total Papers: 5772
Unique Paper IDs: 5772
✅ No duplicate IDs
Track Distribution
Main Track: 5,275 papers (91.4%)
Datasets and Benchmarks Track: 497 papers (8.6%)
Award Distribution
Poster: 4,949 papers (85.7%)
Oral: 84 papers (1.5%)
Spotlight: 739 papers (12.8%)
Track × Award… See the full description on the dataset page: https://huggingface.co/datasets/huyxdang/neurips-2025-papers.IGF-Bench
IGF-Bench: Indoor Geometric Fidelity Benchmark
Anonymous mirror for NeurIPS 2026 Evaluations and Datasets Track double-blind review.
The de-anonymised author/maintainer information will replace this header at camera-ready.
IGF-Bench is the first benchmark for evaluating structural-level geometric fidelity of conditionally generated indoor scene images, going beyond perceptual metrics like FID and LPIPS. It pairs 3,600 calibrated synthetic ground-truth views with 21,600 generated… See the full description on the dataset page: https://huggingface.co/datasets/igfbench-neurips2026/IGF-Bench.NeurIPS-2023-2025
NeurIPS 2023–2025 Peer Review Dataset
Structured peer-review data for 13,171 NeurIPS submissions (2023–2025), collected via the OpenReview API. Each paper entry includes acceptance decisions, full reviewer text, and an anonymized parsed version of the paper itself.
Note on selection bias. NeurIPS authors are not required to make rejection reviews public, and the vast majority do not. As a result, this dataset contains roughly 95% accepted papers. The true NeurIPS acceptance rate is… See the full description on the dataset page: https://huggingface.co/datasets/djroytburg/NeurIPS-2023-2025.CAA
Clinical Agent Annotator (CAA)
A clinician-in-the-loop benchmark for long-horizon medical LLM agents:
333 KG-grounded clinical tasks, multi-gate evaluation across diagnosis,
required tool use, parameterised actions, and must-ask history-taking
topics, plus the full harbor evaluation harness so you can re-run
every number locally.
This repository bundles three artifacts:
Task corpus — 333 clinician-approved tasks (and the 290-task
authoring set the case study trains on).… See the full description on the dataset page: https://huggingface.co/datasets/anon-caa-neurips/CAA.CTSpinoPelvic1K
CTSpinoPelvic1K
A fused spine + pelvis 3D CT segmentation dataset built by patient-level
crosswalk between three public sources:
TCIA CT COLONOGRAPHY — DICOM CT volumes (prone + supine per patient)
CTSpine1K (COLONOG subset) — VerSe-convention vertebral label masks
CTPelvic1K dataset2 — sacrum + bilateral hip label masks
Annotations are placed onto the TCIA CT volume with the highest bone
coverage (HU > 200), separately per anatomy. For ~650 patients both
annotations land on the… See the full description on the dataset page: https://huggingface.co/datasets/anonymous-neurips-ED/CTSpinoPelvic1K.crychic-dafny-acsl
CRYCHIC Dafny-to-ACSL-C Verified Translation Benchmark
This anonymized review artifact accompanies the NeurIPS 2026 Evaluations and Datasets submission:
CRYCHIC: A Universal Framework for Cross-Language Verified Code Translation.
CRYCHIC translates verified Dafny programs into C programs annotated with ACSL specifications, then checks the generated artifacts with Frama-C WP. This release contains the 1,679 fully verified Dafny/C+ACSL pairs used as the positive benchmark corpus.
The… See the full description on the dataset page: https://huggingface.co/datasets/neurips2026-crychic/crychic-dafny-acsl.multisource-memory-benchmark
Multi-Source Memory Benchmark
Status — anonymous artefact for double-blind review (NeurIPS 2026 Evaluations & Datasets Track).
Author identities, organisations, and funders are intentionally withheld until the review period concludes.
A diagnostic testbed for selective question-answering (ANSWER / SKIP) over conflicting multi-source personal memory.
Each persona has five evidence streams projected from a single latent event table with known, controlled per-source distortions… See the full description on the dataset page: https://huggingface.co/datasets/anon-neuripsed26/multisource-memory-benchmark.LatentMD
LatentMD
Benchmarking Markdown Boundary Failures in LLM-Generated Text — NeurIPS 2026 E&D Track submission.
This repository hosts the dataset artifact for LatentMD: the 4,179 prompts that constitute the benchmark, ~37,000 reference responses from 9 frontier models, and illustrative output samples. The accompanying evaluation code (CLI, metric definitions, statistical tests) lives in a separate code repository on GitHub under MIT.
What LatentMD measures
LLM Markdown… See the full description on the dataset page: https://huggingface.co/datasets/latentmd-neurips26/LatentMD.therapyjudgebench
TherapyJudgeBench
An expert-annotated dialogue bank for validating and calibrating LLM-based judges of multi-turn CBT-style therapy conversations. The benchmark accompanies the THERAPYGYM submission to the NeurIPS 2026 Evaluations & Datasets Track.
Anonymous release for double-blind review. Author identity will be revealed upon acceptance.
What It Is and What It Is Not
It is a calibration set for therapy-judge LLMs: 116 simulated patient–therapist dialogues, each rated… See the full description on the dataset page: https://huggingface.co/datasets/neurips-ed-2026-sub3717/therapyjudgebench.NeurIPS-LLM-data
🤖 We curated this dataset for NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day.
🚀 Our Birbal-7B-V1 fine-tuned on this dataset achieved 🏆 first rank 🏆 in the competition.
Here is high-level diagram of our data preparation strategy:
Natural Instructions Dataset Preparation
Natural Instructionsdataset is a community effort to create a large collection of tasks and their natural language definitions/instructions. As show in above diagram, we sample from… See the full description on the dataset page: https://huggingface.co/datasets/upaya07/NeurIPS-LLM-data.ego-mistake-corrections
Ego Mistake Corrections Benchmark (Ego-MC-Bench)
Description
This dataset contains cooking videos with timestamped instruction and feedback for task guidance.
Each row corresponds to one video and provides aligned lists of utterance text, utterance type, and timestamp.
Dataset Details
Release files:
annotations/annotations.json
videos/*.MP4
Release statistics:
Total videos: 40
Total released annotations: 1,597
Text type counts in annotations_release.json:… See the full description on the dataset page: https://huggingface.co/datasets/neuripsedtracksub/ego-mistake-corrections.anon_submission_neurips
Agent Abstention Benchmark — Anonymous NeurIPS 2026 D&B Submission
A benchmark for evaluating when LLM agents should refuse to act versus proceed with tool use.
The dataset contains 263 paired task instances across 8 abstention scenarios:
ambiguous_action_specification
conflicting_constraints
conflicting_evidence
critical_tool_failure
emergent_risk_discovery
high_stakes_action
insufficient_tool_capability
missing_critical_parameter
Pairing structure
Each task is… See the full description on the dataset page: https://huggingface.co/datasets/anonSubmission678/anon_submission_neurips.cga-bench
CGA-Bench Hugging Face Collection
This dataset repo is a collection index for the nine reviewer-facing CGA-Bench dataset descriptors used in the NeurIPS 2026 E&D submission.
Included configs
overview: collection-level summary row spanning the full benchmark release
main_corpus: 19,062-episode primary evaluation corpus
source_grounded: source-grounded SGSC subset
graph_anchored: graph-anchored SGSC subset
profile_expanded: profile-expanded SGSC subset
auto_expanded: 76… See the full description on the dataset page: https://huggingface.co/datasets/cga-bench-neurips26/cga-bench.eda-bench-raw-provenance
EDA Bench raw-extension provenance
This public record documents a private EDA Bench raw extension. It contains no raw designs, source files, account data, or personal information.
The private extension contains 263,257 payload files totaling 61,558,050,574 bytes. Its immutable inclusion manifest, privacy receipt, archive, remote restore, and source-to-restore Git executable-bit parity were verified before this record was prepared.
provenance.json contains content hashes and… See the full description on the dataset page: https://huggingface.co/datasets/eda-bench-neurips-2026/eda-bench-raw-provenance.eda-bench-public-provenance
EDA Benchmark Public Provenance
This dataset records aggregate provenance for the privacy-redacted EDA benchmark extension. It contains no payload files, credentials, personal data, browser data, customer material, or benchmark answers. The raw benchmark evidence remains private while its release rights are reviewed.
neurips-2025-paperswithcode-artifactsformal-anytime-valid-stats
Formal-AVS: A Lean Benchmark for Anytime-Valid Confidence-Sequence Theorem Proving
60 Lean 4 theorem targets on anytime-valid confidence sequences across four families (Howard-Ramdas, betting, Whitehouse vector, asymptotic CLT).
Benchmark Structure
60 targets grouped into tiers T0-T3 (pre-evaluation) and categories T4-T5 (empirical)
7 drafters evaluated across single-shot, agentic, and unbounded modes
14 Aristotle sessions (unbounded refinement)
Headline Results… See the full description on the dataset page: https://huggingface.co/datasets/neurips-2026-avs-bench/formal-anytime-valid-stats.LegalReasoningBench
LegalReasoningBench
A benchmark of structured gold-standard annotations for European Court of
Human Rights (ECtHR) Article 11 (freedom of assembly) cases — covering
case-level conclusions, six doctrinal element conclusions, and
fact-to-element mappings (List 1 / List 2 / List 3).
Released alongside an anonymous NeurIPS 2026 submission.
Quick start
from datasets import load_dataset
ds = load_dataset("neurips-ed-2026-submission-2998/LegalReasoningBench", split="test")… See the full description on the dataset page: https://huggingface.co/datasets/neurips-ed-2026-submission-2998/LegalReasoningBench.ego-mistake-corrections-subsampled
Ego Mistake Corrections Benchmark (Ego-MC-Bench) - Random Sample
Release Files
annotations/annotations.json
videos/*.MP4
sample_manifest.json
Sample Statistics
Total videos: 7
Total annotations: 253
Selected videos:
GX010021
GX010029
GX010033
GX010037
GX010049
GX010053
GX010063
Text type counts:
Feedback: 54
Instruction: 85
Instruction-Clarification: 10
Success: 82
User-Comment: 12
User-Question: 10
Data Format
Each row in… See the full description on the dataset page: https://huggingface.co/datasets/neuripsedtracksub/ego-mistake-corrections-subsampled.ego-counterfactual-mistakes
Ego Counterfactual Mistakes (Ego-CoMist)
Description
This synthetic dataset contains mistake-intervention annotations for interactive cooking guidance. Each row contains video segment with instruction/feedback text pairs and their timestamps.
Dataset Details
Files:
annotations.json
Release statistics:
Total rows: 25,087
Unique videos (dataset + video_id): 1,110
Rows by source dataset:
CaptainCook4D: 4,969
Ego4D: 13,847
Ego-Exo4D: 6,271
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/neuripsedtracksub/ego-counterfactual-mistakes.verbose-listopsvisual_ai_at_neurips2025_jina
Dataset Card for Voxel51/visual_ai_at_neurips2025
This is a FiftyOne dataset with 1134 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("harpreetsahota/visual_ai_at_neurips2025_jina")
# Launch the App
session =… See the full description on the dataset page: https://huggingface.co/datasets/harpreetsahota/visual_ai_at_neurips2025_jina.neurips-2025-audit-poolbeyond-n-grams
Beyond N-Grams (BNG)
Dataset Summary
This dataset is designed to study language trends over time. It combines multi-source metadata, aggregated review signals, and LLM-generated features to enable hypothesis-driven and exploratory research on how narrative and reception characteristics evolve over time.
The dataset does not contain original book text. Instead, it uses LLM-generated proxy content and derived features to approximate semantic and evaluative… See the full description on the dataset page: https://huggingface.co/datasets/annon-neurips-2026/beyond-n-grams.AHaBenchneurips2026-epistemic-honesty
Hard Layer V3: Epistemic Honesty Benchmark for Medical LLMs
Dataset Description
Hard Layer V3 is a 100-question benchmark designed to measure epistemic honesty in medical large language models — whether models explicitly acknowledge uncertainty when confronted with fabricated medical entities, ambiguous thresholds, and knowledge boundaries.
Unlike traditional medical QA benchmarks that focus on accuracy, this benchmark evaluates whether models can appropriately respond… See the full description on the dataset page: https://huggingface.co/datasets/vmti/neurips2026-epistemic-honesty.shopguru-neurips-2026visual_ai_at_neurips2025_nomic
Dataset Card for Voxel51/visual_ai_at_neurips2025
This is a FiftyOne dataset with 1134 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("harpreetsahota/visual_ai_at_neurips2025_nomic")
# Launch the App
session =… See the full description on the dataset page: https://huggingface.co/datasets/harpreetsahota/visual_ai_at_neurips2025_nomic.slaying
SLAyiNG: A Diverse and Community-validated Dataset of Queer Slang
Dataset structure
Each entry of Slaying contains:
id
term: A queer slang term. Sources for these terms are: GSSO, lgbtDB, Wiktionary, and the Chew Inclusive Terminology Glossary.
definition: Definition for the slang term. Some terms have multiple valid definitions -- for those cases, the definitions are split with bullet points.
sentence: Sentence using the term according to the given definition.… See the full description on the dataset page: https://huggingface.co/datasets/slaying-neurips-submission/slaying.