datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
CodeSec-Pairs
CodeSec-Pairs
CodeSec-Pairs is a dataset of matched safe and vulnerable Python code pairs. Each
pair implements the same task but differs in whether it contains a security
vulnerability. Vulnerability labels come from CodeQL static analysis. The dataset is
built to study and steer the internal mechanisms that distinguish safe from vulnerable
code generation in LLMs.
Dataset Details
Each record pairs a CodeQL-clean safe_code with a CodeQL-flagged vuln_code for the… See the full description on the dataset page: https://huggingface.co/datasets/haaao821/CodeSec-Pairs.29K_Python_Docstring_Pairs
29K High-Quality Python Docstring Pairs
Author: Michael Hernandez (XxCotHGxX)License: CC BY 4.0Cleaned from: XxCotHGxX/242K_Python_Docstring_Pairs
Overview
A curated, high-quality subset of Python function–docstring pairs for use in code documentation generation, docstring completion, and code understanding tasks.
The original 242K dataset was scraped from open-source Python repositories but contained a significant proportion of functions without docstrings (84% of… See the full description on the dataset page: https://huggingface.co/datasets/XxCotHGxX/29K_Python_Docstring_Pairs.kana-kanji-pairs
kana-kanji-pairs
Japanese kana-to-kanji conversion candidate dataset.
Overview
Metric
Value
Total pairs
1,124,675
File size
~112MB
Format
JSONL
Candidate Distribution
Candidates
Entries
%
n>=2
363,708
32.3%
n>=5
40,929
3.6%
n>=10
9,401
0.8%
n>=20
2,448
0.2%
n>=100
34
<0.1%
max
259
-
Data Sources
Source
Entries
Description
mozc
753,628
Google mozc dictionary
jmdict
221,228
JMdict… See the full description on the dataset page: https://huggingface.co/datasets/katsukiono/kana-kanji-pairs.yoruba-normalization-pairs
Normalization pairs dataset
What this is
24,475 pairs of Yorùbá text, each a corrupted form next to its canonical form, labelled by corruption type. I built it for testing orthographic normalization code.
The library
This dataset was built alongside yotext, a Python library for Yorùbá orthographic normalization and diacritic restoration. The library is on PyPI at https://pypi.org/project/yotext/ and the source is at… See the full description on the dataset page: https://huggingface.co/datasets/adedejimakinde/yoruba-normalization-pairs.childes-engUK-conversational-pairs
CHILDES Eng-UK Conversational Pairs
Curated naturalistic parent-child conversational pairs extracted from the
English-UK collection of CHILDES (MacWhinney, 2000), with a held-out test
set of 5 complete child histories that no model in the accompanying paper
has seen during training.
Dataset Summary
278,458 conversation pairs total across train, validation, and test
Train: 250,757 pairs from 2,784 transcripts
Validation: 13,197 pairs (in-distribution, sampled from… See the full description on the dataset page: https://huggingface.co/datasets/nshah-fbcs/childes-engUK-conversational-pairs.agillm43-orpo-pairs
AGILLM4.3 ORPO preference pairs
Synthetic {prompt, chosen, rejected} preference pairs for single-stage ORPO
post-training of the AGILLM4.3 base model. Generated with GLM-5.2.
Two preference axes:
Correctness — chosen is right, rejected is fluent-but-wrong. Categories:
capitals, factual QA, arithmetic, worked-step math, science, commonsense,
definitions, short reasoning. Targets the base model's measured weaknesses
(e.g. the "capital of X -> Paris" over-association; word-salad… See the full description on the dataset page: https://huggingface.co/datasets/MarxistLeninist/agillm43-orpo-pairs.SlimOrca-Llama-3-Preference-DPO-Pairs
SlimOrca-Llama-3-Preference-DPO-Pairs
This dataset is based on instructions of SlimOrca-Dedup-Alpaca, with Llama-3 generated response to form a preference dataset.
ALIA-es-biomedical-pairs
Dataset Introduction
The ALIA Spanish Biomedical Retrieval Pairs Corpus contains tabular instances designed to train and evaluate retrieval-oriented models (e.g., dense retrievers / embedding encoders) using passage-grounded query data produced with a Qwen3-style prompting workflow integrated in the ALIA encoders pipeline.
It preserves provenance to the original document and chunk while exposing controls such as question type and difficulty (ranging from high_school to phd level).… See the full description on the dataset page: https://huggingface.co/datasets/SINAI/ALIA-es-biomedical-pairs.shlyokavitsa-pairs
Shlyokavitsa → Cyrillic restoration pairs
210,236 (Latin, Cyrillic) phrase pairs for restoring shlyokavitsa (Bulgarian typed on a
Latin keyboard) back into Cyrillic. Built from Bulgarian Wikipedia, so it can be shared
under the same licence as its source.
{"latin": "sreshta se na dalbochina okolo", "cyrillic": "среща се на дълбочина около", "n_words": 5, "page_id": 1041}
Filed under translation because that is the closest category the Hub offers, but the task is
script… See the full description on the dataset page: https://huggingface.co/datasets/glassbox/shlyokavitsa-pairs.french_orca_dpo_pairs
Dataset Card for french_orca_dpo_pairs
This dataset offers a french translation of the 12k DPO Intel/orca_dpo_pairs pairs made from Open-Orca/OpenOrca.
Dataset Card Contact
ntnq
ALIA-es-cultural-heritage-pairs
Dataset Introduction
The ALIA Spanish Cultural and Heritage Retrieval Pairs Corpus contains tabular instances designed to train and evaluate retrieval-oriented models (e.g., dense retrievers / embedding encoders) using passage-grounded query data produced with a Qwen-style prompting workflow integrated in the ALIA encoders pipeline.
It preserves provenance to the original document and passage while exposing controls such as question type and difficulty (ranging from high_school… See the full description on the dataset page: https://huggingface.co/datasets/SINAI/ALIA-es-cultural-heritage-pairs.warehouse-dpo-preference-pairs
Warehouse Short-Order DPO Preference Pairs
Dataset Description
This dataset contains {prompt, chosen, rejected} preference pairs for
training a warehouse short-order assistant with Direct Preference
Optimization (DPO). Each pair asks a real warehouse-inventory question
(stockout risk, backorders, KPI summaries, why a warehouse is failing
fulfillment - at a single-warehouse, tier, region, or dataset-wide
comparison level) grounded in real tool-call output… See the full description on the dataset page: https://huggingface.co/datasets/EnRaoufi/warehouse-dpo-preference-pairs.bash-agent-grpo-pairs
Bash Agent GRPO Pairs
Single-turn (intent → shell command) pairs for training a small, local, Claude-Code-style
bash agent with SFT or GRPO. Each record pairs a natural-language objective with exactly
one verifiable bash command, framed as a single-tool bash(command, description) call.
The dataset is designed to be rewardable: the ground-truth command is a deterministic target,
so a shell-equivalence reward (canonical program + flag set + argument comparison) can score… See the full description on the dataset page: https://huggingface.co/datasets/adeelahmad/bash-agent-grpo-pairs.intel_orca_dpo_pairs_deGerman translation of Intel/orca_dpo_pairs
Using azureml for translation and hermeo-7b for rejected answers.
codeq-debugbench-dpo-pairs
codeq-debugbench-dpo-pairs
Self-generated preference pairs used to train the CodeQ iterative DPO
pipeline on top of Qwen/Qwen2.5-Coder-7B-Instruct. Each pair consists of
a chosen and rejected response to a DebugBench debugging prompt, where
preferences are derived from MCTS rollouts scored by a unit-test verifier.
Files
File
Rows
Description
round1.jsonl
1515
Raw Round 1 preference pairs (reference = base model).
round1_filtered.jsonl
936
Round 1 after… See the full description on the dataset page: https://huggingface.co/datasets/tathadn/codeq-debugbench-dpo-pairs.Pashto-100k-Pairs
Qehwa AI - Pashto 100K Fine-Tuning Dataset
Overview
Qehwa AI presents a large-scale Pashto instruction tuning dataset containing 100,000+ high-quality instruction-response pairs designed for supervised fine-tuning, conversational AI, and downstream NLP tasks.
This dataset was created to advance AI research for the Pashto language, a significantly underrepresented low-resource language spoken by millions worldwide. The dataset covers more than 20 diverse domains and is… See the full description on the dataset page: https://huggingface.co/datasets/junaid008/Pashto-100k-Pairs.dfm12-norwegian-inclusive-nb-samtale-pairs
dfm12-norwegian-inclusive-nb-samtale-pairs
Published accepted-only DFM12 subset. Local audit-snapshot fields describe the pre-publication build, not Hub publication status.
Only completed kept decisions with all three scores at least 4 are included, after deterministic gates.
Automated review is not native-speaker certification. Exclusion metadata contains only IDs/status/errors/scores/reasons, never excluded conversations.
Full native messages and explicit assistant target… See the full description on the dataset page: https://huggingface.co/datasets/schneiderkamplab/dfm12-norwegian-inclusive-nb-samtale-pairs.ALIA-es-legal-administrative-pairs
Dataset Introduction
The ALIA Spanish Legal and Administrative Pairs Corpus, derived from the SINAI/ALIA-es-legal-administrative, contains tabular instances designed to train and evaluate retrieval-oriented models (e.g., dense retrievers / embedding encoders) using passage-grounded query data produced with a Qwen3-style prompting workflow integrated in the ALIA encoders pipeline.
It preserves provenance to the original document and chunk while exposing controls such as question type… See the full description on the dataset page: https://huggingface.co/datasets/SINAI/ALIA-es-legal-administrative-pairs.dfm12-norwegian-nb-samtale-pairs
dfm12-norwegian-nb-samtale-pairs
Published accepted-only DFM12 subset. Local audit-snapshot fields describe the pre-publication build, not Hub publication status.
Only completed kept decisions with all three scores at least 4 are included, after deterministic gates.
Automated review is not native-speaker certification. Exclusion metadata contains only IDs/status/errors/scores/reasons, never excluded conversations.
Full native messages and explicit assistant target indices are… See the full description on the dataset page: https://huggingface.co/datasets/schneiderkamplab/dfm12-norwegian-nb-samtale-pairs.deepshopper-reward-pairs
DeepShopper Reward pairwise-preference data
(need, chosen=gold outfit, rejected=corrupted outfit, neg_type) pairs for Bradley-Terry
reward training. 93,020 train / 63,584 test, balanced over 5 corruption types:
gender_flip, item_swap, duplicate_role, count_drop, cross_need. Built (scripts/build_reward_pairs.py)
from the gold AMZ bundles via the frozen deepshopper-mapper-reward-splits.
Trains flavianv/qwen4b-reward-pairwise-v1. Code: https://github.com/clijo/reco-rl (branch… See the full description on the dataset page: https://huggingface.co/datasets/flavianv/deepshopper-reward-pairs.oasst2_french_dpo_pairs
Dataset Card for oasst2_french_dpo_pairs
This dataset was created from OpenAssistant/oasst2 by keeping only the french data and producing dpo pairs with their rank.
Dataset Card Contact
ntnq
claudish-pairs
Claudish Pairs
The first open parallel corpus of English ↔ Claudish — the characteristic prose
style of Claude and Claude Code. 10,227 pairs, each an English text and its Claudish
restyling, authored and quality-controlled for faithfulness.
This is the v3 training set of
adamrotmil/claudish-style-adapter;
pipeline code at
github.com/adamrotmil/claudish-style-adapter.
Fields
Field
Meaning
english
source text (plain English)
claudish
the restyling… See the full description on the dataset page: https://huggingface.co/datasets/adamrotmil/claudish-pairs.pashto-stf-grammar-pairs
Pashto SFT Grammar Pairs
Dataset Description
Pashto SFT Grammar Pairs is a native-speaker-curated collection of Pashto question–answer pairs focused on Pashto grammar (ګرامر), covering topics such as noun gender, number, case (فاعلي، مفعولي، اضافي), adjective agreement, pronouns, verb conjugation, sentence structure (SOV word order), and enclitics/suffixes.
The dataset is formatted in the {"messages": [...]} chat-template style used by modern SFT pipelines (TRL… See the full description on the dataset page: https://huggingface.co/datasets/nassimjp/pashto-stf-grammar-pairs.Pashto-Clean-100k-Pairs.QA
Pashto‑Clean‑100k‑Pairs.QA
A curated collection of 100,000 Pashto question–answer pairs, cleaned and normalized for general‑purpose Pashto NLP training.This dataset focuses on broad coverage, topic diversity, and clean formatting, without synthetic reasoning or long‑context generation.
Dataset Summary
Pashto-Clean-100k-Pairs.QA contains short, direct QA pairs across 70+ everyday topics:
Daily life
Community
Education
Work
Nature
Safety
Culture… See the full description on the dataset page: https://huggingface.co/datasets/nassimjp/Pashto-Clean-100k-Pairs.QA.math500-preference-pairs-fable
MATH-500 Preference Pairs (Fable-generated)
500 preference pairs covering all 500 problems of MATH-500, generated by Anthropic's Claude Fable 5 for reward-model training in a math-RLHF project (Qwen2.5-7B, PPO/GRPO on AWS EKS).
Format
{
"idx": 0,
"problem": "Convert the point $(0,3)$ ...",
"chosen": "<complete correct solution with full reasoning>",
"rejected_1": "<plausible-but-wrong solution, error mode A>",
"rejected_2": "<plausible-but-wrong solution… See the full description on the dataset page: https://huggingface.co/datasets/weivzhang/math500-preference-pairs-fable.lmps-challenge-dpo-pairs
LM Playschool Challenge — DPO ablation pairs
Preference data for the study "Teaching or Sharpening? An Exploration of the Potential of DPO
Post-Training for Dialogue Games", which post-trains Qwen3.5 models on the
clembench /
Playpen dialogue-game benchmark.
The paper presenting the results of the study conducted using this dataset will be released upon acceptance.
The question behind the study is whether DPO is suited to teach an LLM new skills, or whether it
mainly sharpens… See the full description on the dataset page: https://huggingface.co/datasets/Makaco/lmps-challenge-dpo-pairs.iac-drift-pairs
IaC Drift Pairs
Declared Infrastructure-as-Code state → drifted live state → structured diff, cause, risk, and fix.
Configuration drift — the gap between what your Terraform/Pulumi/CloudFormation
declares and what actually exists in the cloud — is one of the most expensive and
security-critical problems in cloud operations. Yet there is no public, labeled
dataset that pairs a declared state with its drifted counterpart and the
root cause, risk, and reconciliation. This dataset… See the full description on the dataset page: https://huggingface.co/datasets/Snaseem2026/iac-drift-pairs.Kishor_V2_53K_LLM_Prompt-Response_Pairs
Kishor V2: 53K Prompt-Response Dataset
Kishor V2 is a diverse and compact dataset curated for training small to medium-sized language models. It includes 53,000 structured prompt-response pairs across multiple domains to simulate human-like dialogue, reasoning, and general intelligence.
📦 File
KishorV2_dataset.jsonl: Main dataset in JSON Lines format.
📂 Format
Each line is a JSON object with:
{
"type": "qa" | "dialogue" | "quote" | "fact" | "reasoning" |… See the full description on the dataset page: https://huggingface.co/datasets/GODELEV/Kishor_V2_53K_LLM_Prompt-Response_Pairs.codewraith-training-pairs
CodeWraith Training Pairs
Synthetic (Python source code -> technical specification) pairs generated by
teacher LLMs and used to fine-tune the slenk/codewraith-lora-8b
and slenk/codewraith-merged-8b student models.
See the CodeWraith repo for the full
pipeline: collection, teacher generation, AST + LLM-as-Judge verification,
fine-tuning, and evaluation.
Schema
Each JSONL line:
{
"input": "<python source>",
"output": "<markdown technical specification>"… See the full description on the dataset page: https://huggingface.co/datasets/slenk/codewraith-training-pairs.curated-instruction-pairs
Curated Instruction-Following Pairs
A dataset of 1,025 high-quality instruction/completion pairs spanning coding, writing, reasoning, data science, DevOps, security, and creative tasks. Every example features detailed, production-grade responses modeled after real Stack Overflow answers, LeetCode solutions, official documentation, and professional technical writing.
Dataset Description
Each example contains:
Field
Description
instruction
The task or question… See the full description on the dataset page: https://huggingface.co/datasets/hololivefarm/curated-instruction-pairs.