datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
github-top-code
GitHub Top Developer Source Code
A curated dataset of 1.3M+ source code files from GitHub's top ranked developers (2015-2025).
This dataset is based on the top ranked developers from this dataset: https://huggingface.co/datasets/ronantakizawa/github-top-developers
Dataset Summary
1.3M+ source code files from repositories across ~4,700 unique developers
80+ programming languages included (Python, JavaScript, TypeScript, Rust, Go, C/C++, Java, and more)
Source code only —… See the full description on the dataset page: https://huggingface.co/datasets/ronantakizawa/github-top-code.github-codereview
Code Review Dataset
A large-scale dataset of the best human-written code reviews from top GitHub repositories.
Each row captures a moment where a human code reviewer left an inline comment on a pull request, and the author subsequently modified the code in response.
The dataset also includes negative examples — code from the same PRs that passed review without comments — to help models learn when code is acceptable.
This provides a natural signal for training models to:
Generate… See the full description on the dataset page: https://huggingface.co/datasets/ronantakizawa/github-codereview.code-parrot-github-code
GitHub Code Dataset
Dataset Description
The GitHub Code dataset consists of 115M code files from GitHub in 32 programming languages with 60 extensions totaling in 1TB of data. The dataset was created from the public GitHub dataset on Google BiqQuery.
How to use it
The GitHub Code dataset is a very large dataset so for most use cases it is recommended to make use of the streaming API of datasets. You can load and iterate through the dataset with the following… See the full description on the dataset page: https://huggingface.co/datasets/macrocosm-os/code-parrot-github-code.open-github
OpenGitHub
What is it?
This dataset contains every public event on GitHub: every push, pull request, issue, star, fork, code review, release, and discussion across all public repositories. GitHub is the world's largest software development platform, home to over 200 million repositories and the daily work of tens of millions of developers, from individual open-source contributors to the engineering teams behind the most widely used software on earth.
The archive currently… See the full description on the dataset page: https://huggingface.co/datasets/open-index/open-github.github-jupyter-code-to-text
Dataset description
This dataset consists of sequences of Python code followed by a a docstring explaining its function. It was constructed by concatenating code and text pairs
from this dataset that were originally code and markdown cells in Jupyter Notebooks.
The content of each example the following:
[CODE]
"""
Explanation: [TEXT]
End of explanation
"""
[CODE]
"""
Explanation: [TEXT]
End of explanation
"""
...
How to use it
from datasets import load_dataset
ds =… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/github-jupyter-code-to-text.the-stack-github-issues
Dataset Description
This dataset contains conversations from GitHub issues and Pull Requests. Each conversation is comprised of a series of events, such as opening an issue, creating a comment,
or closing the issue, and includes the author's username, text, action, and identifiers such as the issue ID and number.
The dataset, which is mostly in English, has a total size of 54GB and 30.9M files.
Dataset Structure
from datasets import load_dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/the-stack-github-issues.GitHub-Agentic-PR-Dataset
GitHub Agentic PR Dataset
A large-scale dataset of ~2 million GitHub Pull Requests authored by AI coding agents (Claude Code, Cursor, GitHub Copilot, Devin) and human developers — complete with commits, file-level diffs, patches, and bug-fix classification.
The GitHub Agentic PR Dataset is a research-grade corpus for studying how AI coding agents contribute to real-world open-source software, and how their pull requests compare to those written by humans. It pairs 1,959,649 pull… See the full description on the dataset page: https://huggingface.co/datasets/mabujadallah/GitHub-Agentic-PR-Dataset.github-code-fim
Description
Fill-in-the-middle (FIM) code-completion examples mined from public GitHub repositories. Columns: text (FIM-formatted), source, repo, language.
Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is.
Usage
from datasets import load_dataset
ds = load_dataset("PotatoHD/github-code-fim")
github-pr-agent-trajectories
Description
Reconstructed multi-step agent trajectories for resolving public GitHub PRs: a plan of subtasks and per-subtask subagents with scoped files and solution diffs.
Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is.
Usage
from datasets import load_dataset
ds = load_dataset("PotatoHD/github-pr-agent-trajectories")
github-jupyter-text-code-pairsThis is a parsed version of github-jupyter-parsed, with markdown and code pairs. We provide the preprocessing script in preprocessing.py. The data is deduplicated and consists of 451662 examples.
For similar datasets with text and Python code, there is CoNaLa benchmark from StackOverflow, with some samples curated by annotators.
github-pr-resolve
Description
Issue/PR-review -> resolving patch pairs from merged public GitHub PRs. Includes issue_text, pr_title/pr_body, gold_diff, changed_files, threaded review_threads, and base/head SHAs.
Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is.
Usage
from datasets import load_dataset
ds = load_dataset("PotatoHD/github-pr-resolve")
github-code-haskell-file
Dataset Card for "github-code-haskell-file"
Rows: 339k
Download Size: 806M
This dataset is extracted from github-code-clean.
Each row also contains attribute values for my personal analysis project.
12.6% (43k) of the rows have cyclomatic complexity and LOC valued at -1 because homplexity failed in parsing the row's uncommented_code.
github-java-corpus
github-java-corpus
Summary
This dataset contains Java source-code text samples prepared for pretraining.
Repository
TheFinAI/github-java-corpus
Required Columns
Source: dataset name
Date: year
Text: the pure text of each sample
Token_count: the token count computed with tiktoken
Schema
Source (string)
Date (int32)
Text (string)
Token_count (int32)
Construction
The dataset was built from streamed archive processing into… See the full description on the dataset page: https://huggingface.co/datasets/TheFinAI/github-java-corpus.github-code-scala
Dataset Card for "github-code-scala"
This contains just the scala data in github-code-clean. There are 817k samples with a total download size of 1.52GB.
github-top-code1
GitHub Top Developer Source Code
A curated dataset of 1.3M+ source code files from GitHub's top ranked developers (2015-2025).
This dataset is based on the top ranked developers from this dataset: https://huggingface.co/datasets/ronantakizawa/github-top-developers
Dataset Summary
1.3M+ source code files from repositories across ~4,700 unique developers
80+ programming languages included (Python, JavaScript, TypeScript, Rust, Go, C/C++, Java, and more)
Source code only —… See the full description on the dataset page: https://huggingface.co/datasets/emperorfutures/github-top-code1.deprecated-github-code-haskell-function
Dataset Card for "github-code-haskell-function"
Rows: 3.26M
Download Size: 1.17GB
This dataset is extracted from github-code-haskell-file.
Each row has 3 flavors of the same function:
uncommented_code: Includes the function and its closest signature.
function_only_code: Includes the function only.
full_code: Includes the function and its closest signature and comment.
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the… See the full description on the dataset page: https://huggingface.co/datasets/blastwind/deprecated-github-code-haskell-function.Dr-Zeon-Github-Python-Code-Dataset
Luck Spark 1B - High Quality Code Dataset
The first quality-scored, star-agnostic code dataset for training 1B MoE code models.
Unlike The Stack / CodeParrot that filter by stars, this dataset scores every file by its content (0-10). A 2-star well-documented library scores higher than a 10k-star minified file. Continuously updated by an autonomous bot.
Repo: ahmetggg/luck-spark-1b-code-dataset | Bot: github_to_hf_bot.py | License: Permissive only (MIT / Apache-2.0 / BSD /… See the full description on the dataset page: https://huggingface.co/datasets/ahmetggg/Dr-Zeon-Github-Python-Code-Dataset.github_issues
GitHub Pull Request Bug–Fix Dataset
Kaggle url
A curated, high-signal dataset of real-world software bugs and fixes collected from 25 popular open-source GitHub repositories.Each entry corresponds to a single pull request (PR) and pairs contextual metadata with the exact code changes (unified diffs) that fixed the bug.
This dataset is designed for:
Automated program repair
Bug-fix patch generation
LLM-based code and debugging agents
Empirical software engineering research… See the full description on the dataset page: https://huggingface.co/datasets/helloadhavan/github_issues.github-jupyter-parsed
GitHub Jupyter Dataset
Dataset Description
This is a parsed and preprocessed version of GitHub-Jupyter Dataset, a dataset extracted from Jupyter Notebooks on BigQuery. We only keep markdown and python cells and convert the markdown to text. Some heuristics are also applied to filter notebooks with little data and very long or very short cells.
Licenses
Each example has the license of its associated repository. There are in total 15 licenses:
[
'mit'… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/github-jupyter-parsed.github-python-code-fim
github python code fim
Generated from tomekkorbak/python-github-code, limiting context length to 8192.
github-r-repos
GitHub R repositories dataset
R source files from GitHub.
This dataset has been created using the public GitHub datasets from Google BigQuery.
This is the actual query that has been used to export the data:
EXPORT DATA
OPTIONS (
uri = 'gs://your-bucket/gh-r/*.parquet',
format = 'PARQUET') as
(
select
f.id, f.repo_name, f.path,
c.content, c.size
from (
SELECT distinct
id, repo_name, path
FROM `bigquery-public-data.github_repos.files`
where ends_with(path… See the full description on the dataset page: https://huggingface.co/datasets/dfalbel/github-r-repos.python-github-code-instruct-filtered-5k
Dataset Card for "python-github-code-instruct-filtered-5k"
This fine dataset tomekkorbak/python-github-code, filtered by scores greater than 0.03.
Feedback and additional columns generated through OpenAI and Cohere responses.
github-pytorch-issues
Dataset Card for github-pytorch-issues
Dataset Summary
This dataset is a curated collection of GitHub issues from the PyTorch repository. Each entry includes the issue title, body, user, state, labels, comments, and other relevant fields that are useful for tasks such as text classification, semantic search, and question answering.
Supported Tasks and Leaderboards
The dataset supports the following tasks:
Open-domain Question Answering: Given a user query… See the full description on the dataset page: https://huggingface.co/datasets/mayankpuvvala/github-pytorch-issues.github-top-code
GitHub Top Developer Source Code
A curated dataset of 1.3M+ source code files from GitHub's top ranked developers (2015-2025).
This dataset is based on the top ranked developers from this dataset: https://huggingface.co/datasets/ronantakizawa/github-top-developers
Dataset Summary
1.3M+ source code files from repositories across ~4,700 unique developers
80+ programming languages included (Python, JavaScript, TypeScript, Rust, Go, C/C++, Java, and more)
Source code only —… See the full description on the dataset page: https://huggingface.co/datasets/REXX-NEW/github-top-code.github-python-corpushttps://huggingface.co/datasets/jblitzar/github-python/blob/main/README.md
omnimcp_mcp_github_issue_pr_ops_teaser
🔬 INSPECT THE DEEPSEEK-R1 REASONING CHAIN LIVE:
Zero hallucinations. Null syntax errors. 100% AST compiler validated.🌐 Live Interactive Reasoning & Code Inspector: https://emgena.com/trainingslager🎁 Claim your Free Starter Kit (Code: STARTER100): https://emgena.com/trainingslager🏷️ Launch Discount: Get 20 € OFF any 500-incident production suite with code LAUNCH20!
📜 Enterprise Compliance: EU AI Act Articles 50 & 53 certified • 100% DSGVO / GDPR clean • Commercial EULA… See the full description on the dataset page: https://huggingface.co/datasets/emgena/omnimcp_mcp_github_issue_pr_ops_teaser.Github-Codereasoning-sft-github-codereview
reasoning-sft-github-codereview
Converted version of ronantakizawa/github-codereview, filtered to 76,689 high-quality rows (quality_score >= 0.75, excluding none comment type).
Nothing fancy, just reformatted the columns into a standard messages format for SFT/reasoning training. No content was modified or regenerated.
Format
Each row has three columns:
input — list of dicts with role and content (system prompt + user turn containing the reviewer comment and original… See the full description on the dataset page: https://huggingface.co/datasets/AmanPriyanshu/reasoning-sft-github-codereview.dsir-pile-1m-filtered-no-github-or-dm_mathematics
My_Downsampled_Dataset
This dataset contains 1,000,000 examples from timaeus/dsir-pile-13m-filtered-no-github-or-dm_mathematics, downsampled for efficient processing.
Usage
from datasets import load_dataset
dataset = load_dataset("path/to/my_downsampled_dataset")
github-issues
HuggingFace Datasets Repository Issues
Dataset Description
This dataset contains issues and pull requests from the huggingface/datasets repository, collected via the GitHub API. Each entry includes comprehensive metadata about the issue/PR along with all associated comments, making it valuable for studying software development patterns, issue resolution processes, and community interactions in open-source projects.
Dataset Summary
Repository:… See the full description on the dataset page: https://huggingface.co/datasets/helmo/github-issues.