datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
code_contests
Dataset Card for CodeContests
Dataset Summary
CodeContests is a competitive programming dataset for machine-learning. This
dataset was used when training AlphaCode.
It consists of programming problems, from a variety of sources:
Site
URL
Source
Aizu
https://judge.u-aizu.ac.jp
CodeNet
AtCoder
https://atcoder.jp
CodeNet
CodeChef
https://www.codechef.com
description2code
Codeforces
https://codeforces.com
description2code and Codeforces
HackerEarth… See the full description on the dataset page: https://huggingface.co/datasets/deepmind/code_contests.UltraData-Code
UltraData-Code
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series |
📖 Tech Report (Coming Soon) |
🤗 UltraData-Code-L2 Classifier
English | 中文
📚 Introduction
UltraData-Code is a complete implementation of the UltraData L0-L4 tiered data management framework. It covers four code data states from L0 through L3, with each level corresponding to a distinct construction stage. The pipeline starts from approximately 192 million public GitHub… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-Code.codeforces
Dataset Card for CodeForces
Dataset description
CodeForces is one of the most popular websites among competitive programmers, hosting regular contests where participants must solve challenging algorithmic optimization problems. The challenging nature of these problems makes them an interesting dataset to improve and test models’ code reasoning capabilities. This dataset includes more than 10k unique problems covering the very first contests all the way to 2025.… See the full description on the dataset page: https://huggingface.co/datasets/open-r1/codeforces.CodeTraceBenchCodeTraceBench
A Benchmark for Agent Trajectory Diagnosis
CodeTraceBench is a large-scale benchmark of 4,316 agent trajectories with human-verified step-level annotations for evaluating trajectory diagnosis systems. Each trajectory records the full action-observation sequence of a coding agent, annotated with incorrect and unuseful step labels.
Part of the CodeTracer project — a self-evolving agent trajectory diagnosis system.
Dataset Overview… See the full description on the dataset page: https://huggingface.co/datasets/NJU-LINK/CodeTraceBench.Code-Contests-Plus
CodeContests+: A Competitive Programming Dataset with High-Quality Test Cases
Introduction
CodeContests+ is a competitive programming problem dataset built upon CodeContests. It includes 11,690 competitive programming problems, along with corresponding high-quality test cases, test case generators, test case validators, output checkers, and more than 13 million correct and incorrect solutions.
Highlights
High… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance-Seed/Code-Contests-Plus.codeforces-cots
Dataset Card for CodeForces-CoTs
Dataset description
CodeForces-CoTs is a large-scale dataset for training reasoning models on competitive programming tasks. It consists of 10k CodeForces problems with up to five reasoning traces generated by DeepSeek R1. We did not filter the traces for correctness, but found that around 84% of the Python ones pass the public tests.
The dataset consists of several subsets:
solutions: we prompt R1 to solve the problem and produce code.… See the full description on the dataset page: https://huggingface.co/datasets/open-r1/codeforces-cots.opc-fineweb-code-corpus
OpenCoder Dataset
The OpenCoder dataset is composed of the following datasets:
opc-sft-stage1: the sft data used for opencoder sft-stage1
opc-sft-stage2: the sft data used for opencoder sft-stage2
opc-annealing-corpus: the synthetic data & algorithmic corpus used for opencoder annealing
opc-fineweb-code-corpus: the code-related page recalled from fineweb <-- you are here
opc-fineweb-math-corpus: the math-related page recalled from finewebrefineCode-code-corpus-meta: the meta-data… See the full description on the dataset page: https://huggingface.co/datasets/OpenCoder-LLM/opc-fineweb-code-corpus.CodeGraph
CodeGraph
An open-taxonomy, Wikidata-grounded semantic knowledge graph over 142M source files.
CodeGraph annotates 144,910,008 source files from Stack-Edu, across 14 programming languages, along four orthogonal semantic axes — application domains, algorithms (with category and asymptotic complexity), programming paradigms, and design patterns — and grounds the resulting concept vocabulary in Wikidata. The result is a typed property graph of 158.2M nodes and 1.02B edges that… See the full description on the dataset page: https://huggingface.co/datasets/halykoss/CodeGraph.code-alchemy
CodeAlchemy
CodeAlchemy is a synthetic code dataset (~976.6B tokens, ~162M rows) designed for training and evaluating code language models. It consists of 5 training subsets covering a range of code-related tasks, and 2 evaluation subsets. All files are Parquet with zstd compression with on-disk size ~873 GB. Raw source files are not included due to ownership considerations and must be manually fetched as instructed below.
Dataset Statistics
Config… See the full description on the dataset page: https://huggingface.co/datasets/open-alchemy/code-alchemy.arxiv_cplusplus_research_code
Dataset card for ArtifactAI/arxiv_cplusplus_research_code
Dataset Description
https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_cplusplus_research_code
Dataset Summary
ArtifactAI/arxiv_python_research_code contains over 10.6GB of source code files referenced strictly in ArXiv papers. The dataset serves as a curated dataset for Code LLMs.
How to use it
from datasets import load_dataset
# full dataset (10.6GB of data)
ds =… See the full description on the dataset page: https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_cplusplus_research_code.codeparrot-train-more-filter-3.3b-cleanedcodeforces-submissions
Dataset Card for CodeForces-Submissions
Dataset description
CodeForces is one of the most popular websites among competitive programmers, hosting regular contests where participants must solve challenging algorithmic optimization problems. The challenging nature of these problems makes them an interesting dataset to improve and test models’ code reasoning capabilities.
This dataset includes millions of real user (human) code submissions to the CodeForces website.… See the full description on the dataset page: https://huggingface.co/datasets/open-r1/codeforces-submissions.LiteCoder-Terminal-RL-preview
LiteCoder-Terminal-RL-preview
Paper | Code | Blog Post
This dataset contains 602 standardized Harbor terminal environments and was released as part of the paper LiteCoder-Terminal: Scaling Long-Horizon Terminal Environments for Learning Language Agents.
Unlike static text-only instructions, these environments are fully executable and are designed to support the training of terminal-based agents.
Environment Generation Pipeline
The lack of high-quality, executable… See the full description on the dataset page: https://huggingface.co/datasets/Lite-Coder/LiteCoder-Terminal-RL-preview.Codeforces-Python-Submissions
Dataset Card for "Codeforces-Python-Submissions"
More Information needed
code_contests_instruct
Dataset Card for "code_contests_instruct"
The deepmind/code_contests dataset formatted as markdown-instruct for text generation training.
There are several different configs. Look at them. Comments:
flesch_reading_ease is computed on the description col via textstat
hq means that python2 (aka PYTHON in language column) is dropped, and keeps only rows with flesch_reading_ease 75 or greater
min-cols drops all cols except language and text
possible values for language are {'CPP'… See the full description on the dataset page: https://huggingface.co/datasets/BEE-spoke-data/code_contests_instruct.github-codereview
Code Review Dataset
A large-scale dataset of the best human-written code reviews from top GitHub repositories.
Each row captures a moment where a human code reviewer left an inline comment on a pull request, and the author subsequently modified the code in response.
The dataset also includes negative examples — code from the same PRs that passed review without comments — to help models learn when code is acceptable.
This provides a natural signal for training models to:
Generate… See the full description on the dataset page: https://huggingface.co/datasets/ronantakizawa/github-codereview.RefineCode-code-corpus-metaThis dataset consists of meta information (including the repository name and file path) of the raw code data from RefineCode. You can collect those files referring to this metadata and reproduce RefineCode!
Note: Currently, we have uploaded the meta data covered by The Stack V2 (About 50% file volume). Due to complex legal considerations, we are unable to provide the complete source code currently. We are working hard to make the remaining part available.
RefineCode is a high-quality… See the full description on the dataset page: https://huggingface.co/datasets/OpenCoder-LLM/RefineCode-code-corpus-meta.codeparrot-valid-near-deduplication
CodeParrot 🦜 Dataset after near deduplication (validation)
Dataset Description
A dataset of Python files from Github. We performed near deduplication of this dataset split codeparrot-clean-train from codeparrot-clean. Exact deduplication can miss a fair amount of nearly identical files. We used MinHash with a Jaccard threshold (default=0.85) to create duplicate clusters. Then these clusters are reduced to unique files based on the exact Jaccard similarity. Fore more… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/codeparrot-valid-near-deduplication.codeparrot-train-more-filtering
CodeParrot 🦜 Dataset Cleaned and filtered (train)
Dataset Description
A dataset of Python files from Github. It is a more filtered version of the train split codeparrot-clean-train of codeparrot-clean. The additional filters aim at detecting configuration and test files, as well as outlier files that are unlikely to help the model learn code. The first three filters are applied with a probability of 0.7:
files with a mention of "test file" or "configuration file" or… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/codeparrot-train-more-filtering.arxiv_deep_learning_python_research_code_functions_summaries
Dataset Card for "AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries"
Dataset Description
https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries
Dataset Summary
AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries contains summaries for every python function and class extracted from source code files referenced in ArXiv papers. The… See the full description on the dataset page: https://huggingface.co/datasets/AlgorithmicResearchGroup/arxiv_deep_learning_python_research_code_functions_summaries.hf-codegen-v2
Dataset Card for "hf-codegen-v2"
Dataset generated with the code from: https://github.com/sayakpaul/hf-codegen.
code_x_glue_cc_clone_detection_big_clone_bench
Dataset Card for "code_x_glue_cc_clone_detection_big_clone_bench"
Dataset Summary
CodeXGLUE Clone-detection-BigCloneBench dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/Clone-detection-BigCloneBench
Given two codes as the input, the task is to do binary classification (0/1), where 1 stands for semantic equivalence and 0 for others. Models are evaluated by F1 score.
The dataset we use is BigCloneBench and filtered following the paper… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_clone_detection_big_clone_bench.codeforces-problems-7knova-1-1.3b-code-atlas
juiceb0xc0de/nova-1-1.3b-code-atlas
A brain atlas for Smilyai-labs/Nova-1-Standard-1.3B-Preview, a 24-layer Mixture-of-Depths transformer. This is not a chat dataset or a benchmark. It is an internal-mechanics map built by running activations through a corpus of prompts and scoring what each layer, component, head, and feature direction is doing.
If you want to know what depth routing looks like from the inside, where a small model keeps its register machinery, or how a preview… See the full description on the dataset page: https://huggingface.co/datasets/juiceb0xc0de/nova-1-1.3b-code-atlas.wine-qualityexp-pool-repository-code-dolma2-tokenized
Locus EXP Repository Code - Dolma 2 tokenized
Pretokenized experiment pool for reproducible proxy-training runs.
MANIFEST.json is the authoritative schema, provenance, checksums, and train/holdout assignment.
shards/shard-NNNNN/tokens.bin stores little-endian int32 token IDs.
offsets.bin stores little-endian int64 document boundaries.
index.parquet stores document IDs, offsets, and compact filter fields.
metadata.parquet stores complete source metadata and is downloaded only… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/exp-pool-repository-code-dolma2-tokenized.code_contestsHF-datasets version of Deepmind's code_contests dataset, notably used for AlphaGo. 1 row per solution, no test data or incorrect solutions included (only name/source/description/solution/language/difficulty)
chroma_promptsA collection of prompts captioned using Gemma 2b captioning model. These prompts are intended to be used with FLUX Chroma model.
Download .parquet files to your Google Drive and
run them using the .ipynb notebook in this repo
willow-surface-code-detection-events
Willow Surface-Code Detection Events (ingested)
Detection events and logical-observable flips derived from Google's Willow
below-threshold surface-code dataset (Zenodo
10.5281/zenodo.13273331), rotated
surface code at distances 3, 5, 7 in X and Z memory.
Each row is one experimental shot. Detection events are derived from the raw
device measurement records with Stim's measurement-to-detector converter, using
the per-shot sweep bits, and validated to reproduce the dataset's… See the full description on the dataset page: https://huggingface.co/datasets/ShayManor/willow-surface-code-detection-events.codeparrot-train-v2-near-dedup-safe
Dataset Card for "codeparrot-train-v2-near-dedup-safe"
More Information needed