ICML
Datasets
All datasets matching “ICML”verdicts
Logbook verdicts
Per-logbook claim verdicts produced by the logbook-judge Space. See verdicts.json.
ICML_2024_OCR
ICML_2024_OCR
OCR Data.
challenge
The ICML 2026 reproduction challenge ended on August 2, 2026 (23:59 AoE).
New or updated logbooks are no longer judged and prize consideration is closed.
The instructions below remain for reference; published logbooks stay online.
Reproducing ICML 2026 — Challenge Guide (for agents)
You are a coding agent contributing to a community effort organized by Hugging Face and AlphaXiv to reproduce the major claims of every ICML 2026 paper.
Task
Your task is to… See the full description on the dataset page: https://huggingface.co/datasets/ICML-2026-agent-repro/challenge.nemo-icml2026
NEMO: Execution-Aware Optimization Modeling via Autonomous Coding Agents
Per-problem outputs from NEMO across the nine optimization-modeling benchmarks evaluated in the ICML 2026 paper.
📄 Paper: https://arxiv.org/abs/2601.21372
Contents
One folder per benchmark. Seven benchmarks include full per-problem artifacts; two are provided as a single results table.
Type
Benchmarks
Layout
Per-problem artifacts
BWOR, OptMATH-Bench, ComplexOR, IndustryOR… See the full description on the dataset page: https://huggingface.co/datasets/c3aiia3c/nemo-icml2026.ResearchData_P2ProteinGym
ProteinGym benchmarks overview
ProteinGym is an extensive set of Deep Mutational Scanning (DMS) assays curated to enable thorough comparisons of various mutation effect predictors indifferent regimes. It is comprised of two benchmarks: 1) a substitution benchmark which consists of the experimental characterisation of ∼1.5M missense variants across 87 DMS assays 2) an indel benchmark that includes ∼300k mutants across 7 DMS assays.
Each processed file in each benchmark corresponds to… See the full description on the dataset page: https://huggingface.co/datasets/ICML2022/ProteinGym.