datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ICML2024icml-2026
ICML 2026 — the whole conference, in one queryable dataset
10,815 papers (5,872 main-conference + 4,943 workshop) with titles, abstracts, authors, session times, rooms, and PDF links — plus every tutorial, invited talk, and workshop. Scraped from the official ICML 2026 virtual site and OpenReview, organized so an AI agent can answer real questions about the conference in one shot.
📱 Navigating ICML from your phone? Open Claude Code (or any agent), paste this dataset id, and… See the full description on the dataset page: https://huggingface.co/datasets/Drbellamy/icml-2026.ToneWebinars
ToneWebinars
putnam-axiom-dataset-ICML-2025-522
Putnam AXIOM Dataset (ICML 2025 Version)
Note: for questions, feedback, bugs, etc. please open a Huggingface discussion here.
Dataset Summary
The Putnam AXIOM dataset is designed for evaluating large language models (LLMs) on advanced mathematical reasoning skills. It is based on challenging problems from the Putnam Mathematical Competition. This version contains 522 original problems prepared for the ICML 2025 submission.
The ICML 2025 paper is available on OpenReview:… See the full description on the dataset page: https://huggingface.co/datasets/Putnam-AXIOM/putnam-axiom-dataset-ICML-2025-522.ICML2026icml-2026-reproductions
ICML 2026 Agent Reproducibility Challenge — Logbooks
A living mirror of all public reproduction logbooks from the ICML 2026 Agent Reproducibility Challenge.
Agents attempt to reproduce claims from ICML 2026 papers. Each logbook records the reproduction process, evidence, and verdict for each claim.
Structure
├── papers.json # All 6341 ICML 2026 papers (metadata)
├── logbooks.csv # Main index: one row per logbook (agent × paper ×… See the full description on the dataset page: https://huggingface.co/datasets/qy2100/icml-2026-reproductions.project-level-code-completion
BenchName (Project-level code completion)
This is the benchmark for Project-level code completion task as part of the BenchName benchmark.
Each datapoint contains the file for completion, a list of lines to complete with their categories (see the categorization below), and a repository snapshot that can be used to build the context.
All the repositories are published under permissive licenses (MIT, Apache-2.0, BSD-3-Clause, and BSD-2-Clause). The datapoints can be removed upon… See the full description on the dataset page: https://huggingface.co/datasets/icmlbenchname/project-level-code-completion.ICML2025ICML-Accepted-Papersmedmosaic-dataset
MedMosaic Dataset
A comprehensive medical audio question-answering dataset designed for evaluating audio understanding models in clinical and healthcare contexts.
Dataset Description
This dataset contains audio recordings paired with clinical questions and answers across multiple QA types. It is designed to benchmark audio-language models on medical reasoning tasks.
Dataset Structure
The dataset is organized into 7 subfolders, each representing a different QA… See the full description on the dataset page: https://huggingface.co/datasets/icml-anon-submission/medmosaic-dataset.ToneSlavic
ToneSlavic
ICML2022ICML2023ToneSpeak
ToneSpeak
ICML2020ICML2019ThinkSafe-Qwen3-8B-WildGuardcommit-message-generation
BenchName (Commit message generation)
This is the benchmark for the Commit message generation task as part of the
BenchName benchmark.
All the repositories are published under permissive licenses (MIT, Apache-2.0, and BSD-3-Clause). The datapoints can be removed upon request.
How-to
from datasets import load_dataset
dataset = load_dataset("icmlbenchname/commit-message-generation", split="test")
Note that all the data we have is considered to be in the test split.… See the full description on the dataset page: https://huggingface.co/datasets/icmlbenchname/commit-message-generation.ICML2016putnam-axiom-dataset-ICML-2025-522
Putnam AXIOM Dataset (ICML 2025 Version)
Note: for questions, feedback, bugs, etc. please open a Huggingface discussion here.
Dataset Summary
The Putnam AXIOM dataset is designed for evaluating large language models (LLMs) on advanced mathematical reasoning skills. It is based on challenging problems from the Putnam Mathematical Competition. This version contains 522 original problems prepared for the ICML 2025 submission.
The ICML 2025 paper is available on OpenReview:… See the full description on the dataset page: https://huggingface.co/datasets/introvoyz041/putnam-axiom-dataset-ICML-2025-522.ThinkSafe-Qwen3-1.7B-WildGuardICML2021ThinkSafe-R1-Distill-7B-WildGuardICML2013ICML2015full_before_conv_icmll_pub
Iterative vs Recursive Code Pairs
Coding problems sourced from LeetCode and Codeforces, each with a verified
iterative and recursive Python solution. Test cases are timed and binned
by per-problem difficulty (tc_difficulty: easy | medium | hard). Function names
in both solutions are deterministically obfuscated (*_obfuscated columns) for
benchmarks where lexical signal would leak the paradigm.
Columns
id, task_id, source, difficulty, title, description, tags, rating… See the full description on the dataset page: https://huggingface.co/datasets/CLEVDEV/full_before_conv_icmll_pub.ThinkSafe-R1-Distill-8B-WildGuardToneBooks
ToneBooks
library-based-code-generation
BenchName (Library-based code generation)
This is the benchmark for Library-based code generation task as part of the
BenchName benchmark.
The current version includes 150 manually curated instructions asking the model to generate Python code using a particular library.
The samples come from 62 Python repositories.
All the samples in the dataset are based on reference example programs written by authors of the respective libraries.
All the repositories are published under… See the full description on the dataset page: https://huggingface.co/datasets/icmlbenchname/library-based-code-generation.ICML2018