datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
diffpackDiffPack is the bigcode/commitpack dataset except diff'd between the old and new data.
pile-diff_samp-qwen_1.8B-qwen_104M-r0.5This repository contains the refined pre-training corpus from the paper MiniPLM: Knowledge Distillation for Pre-Training Language Models.
Code: https://github.com/thu-coai/MiniPLM
agent-diff-bench
Agent-Diff Bench
Website | Paper | GitHub
Agent-Diff is a benchmarking framework for evaluating agentic Large Language Models (LLMs) on real-world tasks that execute code via external APIs. The benchmark provides access to real API interfaces (Slack, Box, Linear, Google Calendar) while sandboxing the environment in which calls are made and evaluated.
Dataset Summary
The dataset contains 224 tasks utilizing enterprise software workflows, provided with an 80/20… See the full description on the dataset page: https://huggingface.co/datasets/hubertmarek/agent-diff-bench.pi-diff-review
Coding agent session traces for badlogicgames/pi-diff-review
This dataset contains redacted coding agent session traces collected while working on https://github.com/badlogic/pi-diff-review.git. The traces were exported with pi-share-hf from a local pi workspace and filtered to keep only sessions that passed deterministic redaction and LLM review.
Data description
Each *.jsonl file is a redacted pi session. Sessions are stored as JSON Lines files where each line… See the full description on the dataset page: https://huggingface.co/datasets/badlogicgames/pi-diff-review.stable-diffusion-prompts-2.47M
Source
Combined text-only dataset from
poloclub/diffusiondb
Gustavosta/Stable-Diffusion-Prompts
bartman081523/stable-diffusion-discord-prompts
FredZhang7/krea-ai-prompts
For preprocessing methods, please see Fast GPT2 PromptGen.
Python
Download and save the dataset to all_prompts.txt locally.
pip install datasets
import datasets
dataset = datasets.load_dataset("FredZhang7/stable-diffusion-prompts-2.47M")
train = dataset["train"]
prompts = train["text"]
with… See the full description on the dataset page: https://huggingface.co/datasets/FredZhang7/stable-diffusion-prompts-2.47M.flutter-diff-steps-v1
Flutter Codegen: Diff Steps
Synthetic dataset of step-by-step Flutter/Dart widget construction, where each
row is one incremental edit in a sequence: given a goal, the current code, and the
history of steps taken so far, predict the next action (a short description) and
the code change as a search/replace diff hunk.
Built for training and evaluating small language models on iterative, diff-based
code editing -- as opposed to regenerating the whole file at each step. This is
the… See the full description on the dataset page: https://huggingface.co/datasets/bbidpa/flutter-diff-steps-v1.AM-Math-Difficulty-RLFor more open-source datasets, models, and methodologies, please visit our GitHub repository.
We believe that the selection of training data for reinforcement learning is crucial.
To validate this, we conducted several experiments exploring how data difficulty influences training performance.
Our data sources originate from numerous excellent open-source projects, and we sincerely appreciate their contributions, without which our current achievements would not have been possible.… See the full description on the dataset page: https://huggingface.co/datasets/a-m-team/AM-Math-Difficulty-RL.skill-diffs
skill-diffs
Commit-by-commit revision history of agent skills (SKILL.md files) scraped from public GitHub repos. Each record is a (before, after, intent) tuple capturing how a skill was iteratively refined through human feedback.
v0.5 covers 4 platforms — Anthropic Claude, OpenClaw, OpenCode, and Hermes Agent — with PR title/body metadata as richer intent labels, MinHash + semantic clustering for dedup, structural diff_summary for filtering by edit type, aggregate quality_score for… See the full description on the dataset page: https://huggingface.co/datasets/shl0ms/skill-diffs.pi-diff-review
Coding agent session traces for badlogicgames/pi-diff-review
This dataset contains redacted coding agent session traces collected while working on https://github.com/badlogic/pi-diff-review.git. The traces were exported with pi-share-hf from a local pi workspace and filtered to keep only sessions that passed deterministic redaction and LLM review.
Data description
Each sessions/*.jsonl file is a redacted pi session. Sessions are stored as JSON Lines files where… See the full description on the dataset page: https://huggingface.co/datasets/cfahlgren1/pi-diff-review.latent-dna-diffusionStable-Diffusion-Prompts-Deduped-2.008M
Original Dataset by FredZhang7
Deduped from 2,473,022 down to 2,007,998.
Changed anything that had [ prompt text ], ( prompt text ), or < prompt text >, to [prompt text], (prompt text), and <prompt text>.
2 or more spaces converted to a single space.
Removed all "
Removed spaces at beginnings.
diff-xyz
Diff-XYZ
This is a dataset for the paper: Diff-XYZ: A Benchmark for Evaluating Diff Understanding.
Diff-XYZ contains 1,000 real-world code edits sampled and filtered from
the CommitPackFT dataset.Each example provides three components: the original file contents (old_code), the modified contents (new_code), and
multiple diff representations (udiff, udiff-h, udiff-l, and search-replace).
These formats enable evaluation of LLM capabilities on three code editing tasks:
Apply: Given… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/diff-xyz.task518_emo_different_dialogue_emotions
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task518_emo_different_dialogue_emotions
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task518_emo_different_dialogue_emotions.MathVision_with_difficulty_level
MathVision with difficulty level tags
This dataset extends the 🤗 MathVision benchmark by introducing two additional tags: passrate_for_qwen2.5_vl_7b and difficulty_level_for_qwen2.5_vl_7b. Further details are available in our paper The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs.
🚀 Data Usage
from datasets import load_dataset
dataset = load_dataset("JierunChen/MathVision_with_difficulty_level")
print(dataset)… See the full description on the dataset page: https://huggingface.co/datasets/JierunChen/MathVision_with_difficulty_level.linux-kernel-bugfixes-diffs
🐧 Linux Kernel Bugfixes & Patches Dataset (Instruction-Tuned)
📖 Dataset Description
This dataset is a highly curated, instruction-tuned collection of problem-solution pairs extracted directly from the official Linux Kernel Git repository (torvalds/linux). It is specifically designed to train Large Language Models (LLMs) on low-level C programming, kernel architecture, memory management, and security vulnerability patching.
Unlike raw commit histories, this… See the full description on the dataset page: https://huggingface.co/datasets/switlydev/linux-kernel-bugfixes-diffs.2026-08-31-difficult-advice-716-seeds-bundle
da716 seed replicates — training bundle (seeds 42 and 69)
code.tar.gz (trainer + src/ + the two seed configs) beside seed 0's mixture,
byte-identical. scripts/gpu/runpod_train.py up reads both from this one repo.
field
value
experiment
Seed replicates of the da716 arm (Table2 9,284 filtered + difficult-advice-v2 716, 7.16%) so the arm carries training-seed variance like its siblings. da716 was the last arm on a single seed and is the comparison baseline for the… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-31-difficult-advice-716-seeds-bundle.2026-08-04-synthdoc-difficult-advice-9-principles
Difficult-advice SFT corpus — 9-principle constitution (2,203 examples)
Synthetic difficult-advice data: a user faces an ethically ambiguous situation under real
pressure, and the assistant reasons openly about the value at stake before declining the
norm-violating shortcut and offering a legitimate alternative. Built to replicate the
"difficult advice" result from Anthropic's Teaching Claude Why.
field
value
experiment
Difficult-advice SFT corpus for the Teaching… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-04-synthdoc-difficult-advice-9-principles.2026-08-07-surf-synthdoc-difficult-advice-attributes-full
SURF Attributes (Full)
Complete dataset for SURF research and extension.
Paper: Chunky Post-Training (link pending)
Quick Start
For running SURF, use the minimal dataset: LASR-Callum/2026-08-07-surf-synthdoc-difficult-advice-attributes
uv run -m surf.cli.main sweep \
--attributes LASR-Callum/2026-08-07-surf-synthdoc-difficult-advice-attributes \
--rubric rubrics/rebuttal.yaml \
-o results/
Dataset Fields
prompt: The query text… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-07-surf-synthdoc-difficult-advice-attributes-full.2026-08-21-odcv-difficult-advice-principle-scoped-702-eval
ODCV-Bench — difficult-advice generated WITHOUT the full constitution in refinement
Headline: MR 11.5% [6.2, 19.6], severity 0.62, n=130 (2 rollouts x 65 cells).
The constitution-injection ablation. The baseline difficult-advice recipe injects the WHOLE
constitution into exactly two of its five LLM stages, revise_prompts and
revise_responses; this arm's corpus deleted both injections so no stage ever saw more than
one principle at a time. That also withholds the constitution's… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-21-odcv-difficult-advice-principle-scoped-702-eval.2026-08-07-surf-synthdoc-difficult-advice-attributes
SURF Attributes
Minimal dataset for running SURF (Surfacing Unintended Response Failures).
Paper: Chunky Post-Training (link pending)
Usage
uv run -m surf.cli.main sweep \
--attributes LASR-Callum/2026-08-07-surf-synthdoc-difficult-advice-attributes \
--rubric rubrics/rebuttal.yaml \
-o results/
Fields
prompt: The query text
sae_attributes: List of semantic attribute cluster summaries
How it works
Each prompt was… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-07-surf-synthdoc-difficult-advice-attributes.difficulty-aime_2025-generations
Generations Dataset: aime_2025
Paper: LLMs Encode Their Failures: Predicting Success from Pre-Generation ActivationsCode: GitHub
LLM-generated solutions across train/validation/test splits for multiple models.
Columns
Column
Type
Description
problem
str
Problem statement
generated_solutions
list
Generated solutions with scores
success_rate
float
Fraction of correct generations
majority_vote_is_correct
int (0/1)
Whether majority vote is correct
k… See the full description on the dataset page: https://huggingface.co/datasets/CoffeeGitta/difficulty-aime_2025-generations.2026-08-31-difficult-advice-principle-scoped-702-seeds-bundle
chunk-only 702 seed replicates — training bundle (seeds 42 and 69)
code.tar.gz (trainer + src/ + the two seed configs) beside seed 0's mixture,
byte-identical. scripts/gpu/runpod_train.py up reads both from this one repo.
field
value
experiment
Seed replicates so this arm carries training-seed variance. Table2 9,284 filtered + chunk-only difficult advice 702 (7.03%). The rewrite stages never saw the constitution, only their one target principle. Between-seed spread on… See the full description on the dataset page: https://huggingface.co/datasets/dougalldeepmind/2026-08-31-difficult-advice-principle-scoped-702-seeds-bundle.task628_xlwic_word_with_different_meaning_sentence_generation
Dataset Card for Natural Instructions (https://github.com/allenai/natural-instructions) Task: task628_xlwic_word_with_different_meaning_sentence_generation
Additional Information
Citation Information
The following paper introduces the corpus in detail. If you use the corpus in published work, please cite it:
@misc{wang2022supernaturalinstructionsgeneralizationdeclarativeinstructions,
title={Super-NaturalInstructions: Generalization via Declarative… See the full description on the dataset page: https://huggingface.co/datasets/Lots-of-LoRAs/task628_xlwic_word_with_different_meaning_sentence_generation.DiffutronLM-Pretraining-Corpus
DiffutronLM-Pretraining-Corpus
DiffutronLM-Pretraining-Corpus is the comprehensive, filtered Turkish text dataset used during the Continual Pre-training (CPT) phase of the Diffutron language models.
The primary goal of this dataset was to align the cross-lingual representations of a multilingual base encoder (jhu-clsp/mmBERT-base) with the agglutinative complexity and morphological nuances of the Turkish language, without inducing catastrophic forgetting.
📊 Dataset… See the full description on the dataset page: https://huggingface.co/datasets/diffutron/DiffutronLM-Pretraining-Corpus.cbd-diffsae
cbd-diffsae — model-diffing collection data
Trigger-bearing data for SAE-difference / model-diffing on the fr→gv organism. Configs:
collection (4,096) — balanced poison/single/mismatch/clean; the run-1 collection set.
corpus (3,072) — the fresh, disjoint trigger-bearing subset (poison/single/mismatch, no clean).
collection_v2 (12,173) — the run-2 training set: all suitable fr→gv data (every available
poison/single/mismatch + clean from the organism's own train/dev/heldout, plus… See the full description on the dataset page: https://huggingface.co/datasets/Ftm23/cbd-diffsae.diffusion-generated-text
Diffusion-Generated Text Benchmark
17,565 cleaned responses from three diffusion language model families and 21 generation settings
This benchmark supports research on diffusion-generated language, machine-generated text detection, and robustness across model families and decoding configurations. It includes outputs from DiffusionGemma, LLaDA-8B-Instruct, and LLaDA2-mini with varied generation lengths and block sizes.
Benchmark composition
Generator… See the full description on the dataset page: https://huggingface.co/datasets/paoche11/diffusion-generated-text.civitai-stable-diffusion-2.5minspired by thefcraft/civitai-stable-diffusion-337k.
collected using civitai api to get all prompts.
stable_diffusion_female_prompts
stable_diffusion_female_prompts
⚠ PRECAUTION : This dataset contains NSFW and SFW prompts. ⚠
Stable Diffusion female prompts and some informations about the generated images
Dataset Details
Dataset Description
Enhancing the dataset progress is going on and going to be updated frequently.
These Huggingface datasets are used for enhancing the dataset:
FredZhang7/anime-prompts-180K
FredZhang7/stable-diffusion-prompts-2.47M… See the full description on the dataset page: https://huggingface.co/datasets/WoWoWoWololo/stable_diffusion_female_prompts.Code_Opt_Triton
Overview
This dataset, TEEN-D/Code_Opt_Triton, is an extended version of the publicly available GPUMODE/Inductor_Created_Data_Permissive dataset. It contains pairs of original (PyTorch or Triton) programs and their equivalent Triton code (generated by torch inductor), intended for training models in PyTorch-to-Triton code translation and optimization.
The primary modification in this extended version is that each optimized Triton code snippet is paired with both its original source… See the full description on the dataset page: https://huggingface.co/datasets/Teen-Different/Code_Opt_Triton.OpenMath-Difficulty-Annotated
📐 OpenMath-Difficulty-Annotated
🚀 Overview
OpenMath-Difficulty-Annotated is a curated subset of OpenMathInstruct-2 containing 10,176 math problems, enhanced with precise difficulty metadata.
While the original solutions are preserved from NVIDIA's dataset, we employed a 120B Parameter Model (LLM-as-a-Judge) to analyze and grade every single problem on a scale of 1 to 5.
This allows developers of Small Language Models (1B-3B) to filter out "Olympiad-level" noise… See the full description on the dataset page: https://huggingface.co/datasets/HAD653/OpenMath-Difficulty-Annotated.