datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
locus-commit-pool-v1
Locus Commit Pool v1
Native Git history, preserved as replayable software changes
Commit message · complete selected before-state · unified patches · native object IDs · provenance · experimental labels
Locus Commit Pool v1 is a large evidence pool for studying and training on how real software changes. Each document represents one surviving single-parent, multi-file Git commit. It keeps the commit message, the selected files as they existed before the… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/locus-commit-pool-v1.knesset-committees
About
This dataset is derived from raw a/v recordings and human-generated protocols of the Knesset (the Israeli house of representatives) committee sessions as part of the ivrit.ai project.
Consider visiting the preview space for this dataset here
Method
Data dumps from the Knesset contain A/V recordings of committee sessions, alongside human-generated protocols.
We extract the audio stream, abd produce weakly time stamped segmentation of the protocol text (we… See the full description on the dataset page: https://huggingface.co/datasets/ivrit-ai/knesset-committees.git-commits
Themis-Git-Commits
Overview
Themis-Git-Commits is a large-scale dataset of single-file code commits mined from permissively licensed GitHub repositories via the BigQuery GitHub public dataset. The SQL query restricts to repositories under permissive open-source licenses only (MIT, Apache-2.0, BSD-2/3-Clause, ISC, CC0-1.0, EPL-1.0, MPL-2.0, Unlicense, AGPL-3.0, LGPL-2.1, Artistic-2.0). The BigQuery snapshot used contains commits up to early 2022 —… See the full description on the dataset page: https://huggingface.co/datasets/project-themis/git-commits.conventional-commits
Git Diff → Conventional Commit Messages
A dataset of 12,433 (git diff, commit message) pairs scraped from real open-source TypeScript repositories, filtered for quality and formatted for fine-tuning small language models to generate Conventional Commits.
Built as part of a project to fine-tune a local LLM to write commit messages as a prepare-commit-msg git hook. Full write-up: eliotbas.com/projects/commits-fine-tuning
Dataset details
Source… See the full description on the dataset page: https://huggingface.co/datasets/Elib27/conventional-commits.commit-chronicle
📜 CommitChronicle 🔮
This is the dataset for commit message generation (and/or completion), introduced in the paper "From Commit Message Generation to History-Aware Commit Message Completion", ASE 2023.
Its key features:
large-scale and multilingual: contains 10.7M commits from 11.9k GitHub repositories in 20 programming languages;
diverse: avoids restrictive filtering on commit messages or commit diffs structure;
suitable for experiments with commit history: provides metadata… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/commit-chronicle.git-commits-merged
Themis-Git-Commits-Merged
Overview
Themis-Git-Commits-Merged is a large-scale dataset of ~3.98M single-file code commits from permissively licensed GitHub repositories that have been cross-referenced with GHTorrent pull request data to retain only commits that are part of successfully merged, non-reverted pull requests. This provides implicit human validation of each code change — a merge decision by project maintainers confirms the intent and quality of… See the full description on the dataset page: https://huggingface.co/datasets/project-themis/git-commits-merged.commit-chronicle
📜 CommitChronicle 🔮
This is the dataset for commit message generation (and/or completion), introduced in the paper "From Commit Message Generation to History-Aware Commit Message Completion", ASE 2023.
Its key features:
large-scale and multilingual: contains 10.7M commits from 11.9k GitHub repositories in 20 programming languages;
diverse: avoids restrictive filtering on commit messages or commit diffs structure;
suitable for experiments with commit history: provides metadata… See the full description on the dataset page: https://huggingface.co/datasets/Hirunima/commit-chronicle.exp-pool-commit-code-dolma2-tokenized
Locus EXP Commit Code - Dolma 2 tokenized
Pretokenized experiment pool for reproducible proxy-training runs.
MANIFEST.json is the authoritative schema, provenance, checksums, and train/holdout assignment.
shards/shard-NNNNN/tokens.bin stores little-endian int32 token IDs.
offsets.bin stores little-endian int64 document boundaries.
index.parquet stores document IDs, offsets, and compact filter fields.
metadata.parquet stores complete source metadata and is downloaded only for… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/exp-pool-commit-code-dolma2-tokenized.tangled-ccs-commits
Detecting Multiple Semantic Concerns in Tangled Code Commits using Small Language Models
This dataset contains commit data for training and evaluating models on software engineering tasks, specifically focusing on identifying and separating concerns in multi-concern commits.
Every tangled (multi-concern) commit in this dataset is composed exclusively of atomic commits from a single repository — resolving a structural weakness in earlier cross-repo tangles (which were trivially… See the full description on the dataset page: https://huggingface.co/datasets/Berom0227/tangled-ccs-commits.sejm-committee-transcripts
Polish Sejm committee transcripts — full API coverage (terms 9 and 10)
Official committee transcripts ("pełny zapis przebiegu posiedzenia") from the Sejm
of the Republic of Poland, parsed into individually attributed speaker turns.
Scope
Committees: all standing committees with zapis PDFs in the Sejm API.
Terms: 9 (2019-11-14 → 2023-11-09) and 10 (2023-11-14 → 2026-09-17).
Provider and primary source: Kancelaria Sejmu RP, https://api.sejm.gov.pl/… See the full description on the dataset page: https://huggingface.co/datasets/PiotrSty/sejm-committee-transcripts.committed-train
Committed — Conventional Commits dataset
Filtered (diff -> Conventional Commits message) pairs derived from
CommitChronicle,
for fine-tuning small models to write commit messages from a diff. Built by the
Committed project.
Schema
Field
Type
Notes
diff
string
The code diff for a single-file change.
message
string
Normalized Conventional Commits subject line (the training target).
reasoning_trace
string | null
Reserved for v2 (reasoning… See the full description on the dataset page: https://huggingface.co/datasets/marzoukbaig14/committed-train.vaani-gujarati-sft-data
Vaani — Gujarati SFT & Eval Data
Final datasets for the Vaani 110M Gujarati medical SLM.
File
Rows
Purpose
sft_v4.jsonl
~150k
Instruction SFT: general instructions + format skills (JSON, extraction, exact-count lists)
medical_sft_v3.jsonl
~114k
Medical SFT: closed-book MedMCQA-gu + raw-grounded + redacted-grounded + 30% general mix
medmcqa_gu_val.jsonl
4,183
Held-out MedMCQA-gu validation split (eval only; disjoint from SFT)
The pretraining corpus is… See the full description on the dataset page: https://huggingface.co/datasets/pratham-commits/vaani-gujarati-sft-data.commit-message-generationA dataset for training models to generate descriptive commit messages from git diffs. Contains 13,811 examples with git diffs and corresponding commit messages in chat format suitable for instruction tuning.
Dataset Statistics
Number of examples: 13,811
Average text length: 960 characters
Loading the Dataset
from datasets import load_dataset
dataset = load_dataset("hks350d/commit-message-generation")
Example Usage
# Load the dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/hks350d/commit-message-generation.openbsd-commits-alpaca
OpenBSD Commit History — Alpaca Format (v1)
Fine-tuning dataset derived from the full commit history of the
OpenBSD src repository, structured
for instruction fine-tuning in Alpaca format.
Task: given a unified diff, generate the commit message.
Dataset Summary
Field
Value
Examples
103,383
Size
~202 MB
Format
Alpaca JSONL
Date range
2000-01-01 → present
Source
openbsd/src (GitHub mirror)
License
ISC
Format
Each line is a… See the full description on the dataset page: https://huggingface.co/datasets/ajsbsd/openbsd-commits-alpaca.pr-poet-commits
PR Poet Commit Poems
One thousand public Git repository commit messages paired with synthetic four-line poems generated for PR Poet.
Each record contains:
repo: the source GitHub repository
commit: the commit message
target: the generated poem as line_one through line_four
This is the original generated source set used by the PR Poet training curriculum. It intentionally includes poems that were later filtered out for rhyme or style, so not every example satisfies the final… See the full description on the dataset page: https://huggingface.co/datasets/mkly/pr-poet-commits.hyperswitch-rust-commits-final
Rust Commit Dataset - Hyperswitch
Dataset Description
This dataset contains Rust commit messages paired with their corresponding code patches from the Hyperswitch repository.
Dataset Summary
Total Examples: 2277
Language: Rust
Source: Hyperswitch GitHub repository
Format: Prompt-response pairs for supervised fine-tuning (SFT)
Data Fields
prompt: The commit message describing the change
response: The git patch/diff showing the actual code changes… See the full description on the dataset page: https://huggingface.co/datasets/archit11/hyperswitch-rust-commits-final.hyperswitch-rust-commits
Rust Commit Dataset - Hyperswitch
Dataset Description
This dataset contains Rust commit messages paired with their corresponding code patches from the Hyperswitch repository.
Dataset Summary
Total Examples: 0
Language: Rust
Source: Hyperswitch GitHub repository
Format: Prompt-response pairs for supervised fine-tuning (SFT)
Data Fields
prompt: The commit message describing the change
response: The git patch/diff showing the actual code changes… See the full description on the dataset page: https://huggingface.co/datasets/archit11/hyperswitch-rust-commits.hyperswitch-rust-commitsv5
Rust Commit Dataset - Hyperswitch
Dataset Description
This dataset contains Rust commit messages paired with their corresponding code patches from the Hyperswitch repository.
Dataset Summary
Total Examples: 2277
Language: Rust
Source: Hyperswitch GitHub repository
Format: Prompt-response pairs for supervised fine-tuning (SFT)
Data Fields
prompt: The commit message describing the change
response: The git patch/diff showing the actual code changes… See the full description on the dataset page: https://huggingface.co/datasets/archit11/hyperswitch-rust-commitsv5.bill_committees_us
Dataset Card for "bill_committees_us"
Dataset Summary
Dataset for US Congressional bills with committees information (bill_committees_us). Contains data for bills from the 108th to the 118th Congress, approximately 132,000 documents.
Supported Tasks and Leaderboards
More Information Needed
Languages
English
Dataset Structure
Data Instances
default
Data Fields
id: id of the bill in format(congress number +… See the full description on the dataset page: https://huggingface.co/datasets/dreamproit/bill_committees_us.hyperswitch-rust-commitsv4
Rust Commit Dataset - Hyperswitch
Dataset Description
This dataset contains Rust commit messages paired with their corresponding code patches from the Hyperswitch repository.
Dataset Summary
Total Examples: 328
Language: Rust
Source: Hyperswitch GitHub repository
Format: Prompt-response pairs for supervised fine-tuning (SFT)
Data Fields
prompt: The commit message describing the change
response: The git patch/diff showing the actual code changes… See the full description on the dataset page: https://huggingface.co/datasets/archit11/hyperswitch-rust-commitsv4.hyperswitch-rust-commits-final2
Rust Commit Dataset - Hyperswitch
Dataset Description
This dataset contains Rust commit messages paired with their corresponding code patches from the Hyperswitch repository.
Dataset Summary
Total Examples: 2277
Language: Rust
Source: Hyperswitch GitHub repository
Format: Prompt-response pairs for supervised fine-tuning (SFT)
Data Fields
prompt: The commit message describing the change
response: The git patch/diff showing the actual code changes… See the full description on the dataset page: https://huggingface.co/datasets/archit11/hyperswitch-rust-commits-final2.