datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
gitskills
GitSkills: A Dataset of Agent Skills on GitHub
Paper (arXiv:2608.10906) ·
Sample repository ·
Zenodo DOI: 10.5281/zenodo.21875637
An agent skill is a folder containing a SKILL.md file with instructions for
a language-model agent, optionally accompanied by scripts and reference
files. The agent loads the skill when it judges that a task matches the
skill description. Anthropic introduced the format in October 2025 as an
open specification. Nine months later, skill files in the… See the full description on the dataset page: https://huggingface.co/datasets/mvaccargiu/gitskills.github-codereview
Code Review Dataset
A large-scale dataset of the best human-written code reviews from top GitHub repositories.
Each row captures a moment where a human code reviewer left an inline comment on a pull request, and the author subsequently modified the code in response.
The dataset also includes negative examples — code from the same PRs that passed review without comments — to help models learn when code is acceptable.
This provides a natural signal for training models to:
Generate… See the full description on the dataset page: https://huggingface.co/datasets/ronantakizawa/github-codereview.open-github
OpenGitHub
What is it?
This dataset contains every public event on GitHub: every push, pull request, issue, star, fork, code review, release, and discussion across all public repositories. GitHub is the world's largest software development platform, home to over 200 million repositories and the daily work of tens of millions of developers, from individual open-source contributors to the engineering teams behind the most widely used software on earth.
The archive currently… See the full description on the dataset page: https://huggingface.co/datasets/open-index/open-github.github-diffs-dedupedthe-stack-github-issues
Dataset Description
This dataset contains conversations from GitHub issues and Pull Requests. Each conversation is comprised of a series of events, such as opening an issue, creating a comment,
or closing the issue, and includes the author's username, text, action, and identifiers such as the issue ID and number.
The dataset, which is mostly in English, has a total size of 54GB and 30.9M files.
Dataset Structure
from datasets import load_dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/the-stack-github-issues.github-readme-retrieval-multilingual_beirThis is a copy of https://huggingface.co/datasets/jinaai/github-readme-retrieval-multilingual reformatted into the BEIR format. For any further information like license, please refer to the original dataset.
Disclaimer
This dataset may contain publicly available images or text data. All data is provided for research and educational purposes only. If you are the rights holder of any content and have concerns regarding intellectual property or copyright, please contact us at… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/github-readme-retrieval-multilingual_beir.python-github-codegithub-issuesgithub-repos-metadata-40M
📊 Metadata for 40 million GitHub repositories
A cleaned, analysis-ready dataset with per-repository statistics aggregated from GH Archive events: stars, forks, pull requests, open issues, visibility, language signals, and more. Column names mirror the GH Archive / GitHub API semantics where possible.
GitHub repo: https://github.com/ibragim-bad/github-repos-metadata-40M
Source: GH Archive (public GitHub event stream).
✅ Projects with complementary ideas
GitHub Repo… See the full description on the dataset page: https://huggingface.co/datasets/ibragim-bad/github-repos-metadata-40M.GitHub-Agentic-PR-Dataset
GitHub Agentic PR Dataset
A large-scale dataset of ~2 million GitHub Pull Requests authored by AI coding agents (Claude Code, Cursor, GitHub Copilot, Devin) and human developers — complete with commits, file-level diffs, patches, and bug-fix classification.
The GitHub Agentic PR Dataset is a research-grade corpus for studying how AI coding agents contribute to real-world open-source software, and how their pull requests compare to those written by humans. It pairs 1,959,649 pull… See the full description on the dataset page: https://huggingface.co/datasets/mabujadallah/GitHub-Agentic-PR-Dataset.multiturn_processedgithub-pr-agent-trajectories
Description
Reconstructed multi-step agent trajectories for resolving public GitHub PRs: a plan of subtasks and per-subtask subagents with scoped files and solution diffs.
Derived dataset. Source material retains its original per-item licence (see source/repo columns); treat as other / mixed. Provided as-is.
Usage
from datasets import load_dataset
ds = load_dataset("PotatoHD/github-pr-agent-trajectories")
github-code-haskell-file
Dataset Card for "github-code-haskell-file"
Rows: 339k
Download Size: 806M
This dataset is extracted from github-code-clean.
Each row also contains attribute values for my personal analysis project.
12.6% (43k) of the rows have cyclomatic complexity and LOC valued at -1 because homplexity failed in parsing the row's uncommented_code.
GitScholar
GitScholar: arXiv AI papers, their citations, and their GitHub footprint
GitScholar is a relational, fully timestamped dataset linking AI arXiv papers
to their citation history on Semantic Scholar and to the GitHub repositories that
reference them. It is built for studying, and predicting, how research papers gain
attention over time: every row carries the date on which it became true, so the state
of the whole graph can be reconstructed as of any day between 1991 and… See the full description on the dataset page: https://huggingface.co/datasets/huawei-csl/GitScholar.github-repos-metadata-ge3
GitHub All Repositories Metadata Dataset (>= 3 Stars)
A comprehensive metadata dataset covering 5,289,726 public GitHub repositories with 3 or more stars (>= 3) spanning the history of GitHub from 2008 to 2026.
Data Recency & Snapshot Notice
[!NOTE]
Snapshot Methodology: This dataset combines a comprehensive historical base archive (up to mid-2022) with continuous periodic crawler snapshots (2023 through 2026).
Star Counts & Metrics: Star counts and repository… See the full description on the dataset page: https://huggingface.co/datasets/Mieaz/github-repos-metadata-ge3.github-issues-updated
📊 GitHub Issues Dataset (HuggingFace/datasets Repository)
This dataset contains structured GitHub issues scraped from the huggingface/datasets repository. It is intended for NLP tasks, topic modeling, issue classification, and software engineering research.
📌 Dataset Summary
Repository Source: huggingface/datasets
Scraped via: GitHub REST API v3
Total Issues: ~7,465
Collected On: June 25, 2025
Format: JSONL → loaded via Arrow for Hugging Face
Language: English… See the full description on the dataset page: https://huggingface.co/datasets/rIsHu009/github-issues-updated.github-java-corpus
github-java-corpus
Summary
This dataset contains Java source-code text samples prepared for pretraining.
Repository
TheFinAI/github-java-corpus
Required Columns
Source: dataset name
Date: year
Text: the pure text of each sample
Token_count: the token count computed with tiktoken
Schema
Source (string)
Date (int32)
Text (string)
Token_count (int32)
Construction
The dataset was built from streamed archive processing into… See the full description on the dataset page: https://huggingface.co/datasets/TheFinAI/github-java-corpus.git10m-instances
Git-10M Instance Annotations (v1.0)
101.6 million polygon-level object annotations across 72 geographic regions, anchored to Git-10M satellite tiles at zoom levels 10–18. Each instance is a polygon (building footprint or OSM area feature) with structured OSM tags, projected into the pixel frame of its deepest containing Git-10M tile.
This is an annotations-only release — the satellite imagery itself is in lcybuaa/Git-10M and must be downloaded separately.
What's in… See the full description on the dataset page: https://huggingface.co/datasets/dcher95/git10m-instances.ultrainteract_pair_harvarddeprecated-github-code-haskell-function
Dataset Card for "github-code-haskell-function"
Rows: 3.26M
Download Size: 1.17GB
This dataset is extracted from github-code-haskell-file.
Each row has 3 flavors of the same function:
uncommented_code: Includes the function and its closest signature.
function_only_code: Includes the function only.
full_code: Includes the function and its closest signature and comment.
The heuristic for finding the closest signature and comment follows: If the immediate previous neighbor of the… See the full description on the dataset page: https://huggingface.co/datasets/blastwind/deprecated-github-code-haskell-function.Dr-Zeon-Github-Python-Code-Dataset
Luck Spark 1B - High Quality Code Dataset
The first quality-scored, star-agnostic code dataset for training 1B MoE code models.
Unlike The Stack / CodeParrot that filter by stars, this dataset scores every file by its content (0-10). A 2-star well-documented library scores higher than a 10k-star minified file. Continuously updated by an autonomous bot.
Repo: ahmetggg/luck-spark-1b-code-dataset | Bot: github_to_hf_bot.py | License: Permissive only (MIT / Apache-2.0 / BSD /… See the full description on the dataset page: https://huggingface.co/datasets/ahmetggg/Dr-Zeon-Github-Python-Code-Dataset.Gitruck-LUT-15K
Gitruck LUT 15K
Gitruck LUT 15K is an authorized collection of .cube color lookup tables
prepared for LUT retrieval, similarity, classification, color-science analysis,
and generative modeling. The release contains viewable Parquet metadata,
diagnostic before/after previews, normalized tensors, and lossless raw archives.
Release summary
Item
Count
Source records
15,792
Unique raw byte streams
14,021
Unique normalized LUTs
13,365
Canonical train… See the full description on the dataset page: https://huggingface.co/datasets/Hocassian/Gitruck-LUT-15K.git-diff-Qwen-4B-rolloutsmultiturn_1_2_harvardgithub-python-code-fim
github python code fim
Generated from tomekkorbak/python-github-code, limiting context length to 8192.
llama3-ultrafeedback-reasoning-ReRe-armo-tokenizedllama3-ultrafeedback-reasoning-ReRe-armo-tokenized_harvardllama3-ultrafeedback-reasoning-iter_2-1732268914-tokenized_harvardgithub-issuesllama3-ultrafeedback-reasoning-iter_2-1732268914-tokenized