datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
stack-v3-train
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceCode/stack-v3-train.Beta-Pre-Train-Corpus
Reactive AI / Beta Pre-Train Corpus
Pre-training corpus for RxT-Beta models, created from public & open datasets. Includes high-quality english and polish web crawl data, mathematic and scientific subsets,
and code in different programming languages.
2k subsets are filtered for 1024-2048 tokens, except MegaMath Web Pro and GitHub Code subsets, that were filtered for 512-2048 tokens
Subsets & original datasets
FineWeb-Edu
fineweb-edu-s100 (51.3M examples) - 50% of… See the full description on the dataset page: https://huggingface.co/datasets/ReactiveAI/Beta-Pre-Train-Corpus.OmniRet-train
OmniRet training dataset
OmniRet-train is the training-data release for
OmniRet, a unified retrieval model for
text, image, video, and audio. This card documents the released snapshot for
researchers training or analyzing OmniRet.
Dataset summary
The release contains 6,405,109 query rows and 7,119,841 candidate rows from 30
datasets. It covers 15 retrieval directions across text (T), image (I), video
(V), and audio (A). The OmniRet paper reports this corpus as… See the full description on the dataset page: https://huggingface.co/datasets/chuonghm/OmniRet-train.TrainingData_Stage3
AnchorSR Stage3 · metric-v1.0
直接选择 Small / Large
配置
训练题数
用途
small
1,000,000
先验证答案监督／先验恢复，按新版 Large 联合分布抽样
large
89,801,853
筛选后的完整训练集合，包含 Small 全部样本
from datasets import load_dataset
data = load_dataset('AnchorSR/TrainingData_Stage3', 'small', # 或 large
revision='metric-v1.0', streaming=True)
这是对 scaling-v1.0 的语义筛选与统一任务分类，不是增加新数据源。
Large 从 89,828,269 题保留 89,801,853 题，隔离 26,416 题。
旧标签 scaling-v1.0 / video-v1.0 / large-v1.0… See the full description on the dataset page: https://huggingface.co/datasets/AnchorSR/TrainingData_Stage3.abc_130k_v3_trainThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"observation.state": {
"dtype": "float32",
"shape": [
14
],
"names": [
"left_arm_joint_1",
"left_arm_joint_2",
"left_arm_joint_3",
"left_arm_joint_4",
"left_arm_joint_5"… See the full description on the dataset page: https://huggingface.co/datasets/lerobot/abc_130k_v3_train.OMol25_train
Cite this dataset Levine, D. S., Shuaibi, M., Spotte-Smith, E. W. C., Taylor, M. G., Hasyim, M. R., Michel, K., Batatia, I., Csányi, G., Dzamba, M., Eastman, P., Frey, N. C., Fu, X., Gharakhanyan, V., Krishnapriyan, A. S., Rackers, J. A., Raja, S., Rizvi, A., Rosen, A. S., Ulissi, Z., Vargas, S., Zitnick, C. L., Blau, S. M., and Wood, B. M. OMol25 train. ColabFit, 2025. https://doi.org/10.60732/41666b82
This dataset has been curated and formatted for the ColabFit… See the full description on the dataset page: https://huggingface.co/datasets/colabfit/OMol25_train.humanoid-robots-training-dataset
Dynamic Intelligence — Humanoid Robot Training Dataset
A first-person (egocentric) video dataset of human hand manipulation, designed for training humanoid robot policies via imitation learning. Each episode captures a person performing an everyday household task — folding clothes, moving dishes, opening doors — filmed from a head-mounted iPhone using its built-in LiDAR and depth sensors.
The dataset pairs each video with frame-level 3D hand tracking and camera pose data, giving… See the full description on the dataset page: https://huggingface.co/datasets/DynamicIntelligence/humanoid-robots-training-dataset.codeparrot-train-more-filter-3.3b-cleanedstack-v3-train
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/AlanaSky/stack-v3-train.stack-v3-train
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/CathleenTico/stack-v3-train.train_ctf_eefmtnwx-trainingtrain_ctfstack-v3-train
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/mick260449/stack-v3-train.SWE-Dev-train📝 Paper | 🌐 Github
🤗 SWE-Dev-7B (Qwen-2.5-Coder-7B-Instruct)
🤗 SWE-Dev-9B (GLM-4-9B-Chat)
🤗 SWE-Dev-32B (Qwen-2.5-Coder-32B-Instruct)
🤗 SWE-Dev-train (Training Data)
🚀 SWE-Dev, an open-source Agent for Software Engineering tasks! This repository contains the SWE-Dev-32B model as presented in the paper SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling.
💡 We develop a comprehensive pipeline for creating developer-oriented datasets from GitHub… See the full description on the dataset page: https://huggingface.co/datasets/zai-org/SWE-Dev-train.rag_multilingual_training_negatives
How this dataset was made
We trained on chunks sourced from the documents in MADLAD-400 dataset that had been evaluated to contain a higher amount of educational information according to a state-of-the-art LLM.
We took chunks of size 250 tokens, 500 tokens, and 1000 tokens randomly for each document.
We then used these chunks to generate questions and answers based on this text using a state-of-the-art LLM.
Finally, we selected negatives for each chunk using the similarity from the… See the full description on the dataset page: https://huggingface.co/datasets/lightblue/rag_multilingual_training_negatives.fastdetector-train-stat-train
Auto-generated FastDetector dataset
Fastdetector Train Stat Train
Best detectorGiga EditLens Llama-3.2-3B Score0.7121 TPR @ 1% FPRHardest prompt subsetrewrite0.3324 max detector TPR @ 1% FPRHardest generator configdeepseek-v4.1-flash (Temp: Unknown)0.4579 max detector TPR @ 1% FPR
336,087rows15generator configs4prompt subsets4detectors
01Leaderboard02Model analytics03Distances04Appendix
01Detector leaderboardScore-based detectors ranked by overall AUROC. Thresholds are placed exactly on… See the full description on the dataset page: https://huggingface.co/datasets/G-reen/fastdetector-train-stat-train.stack-v3-train
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/happahhap2026/stack-v3-train.codeparrot-train-more-filtering
CodeParrot 🦜 Dataset Cleaned and filtered (train)
Dataset Description
A dataset of Python files from Github. It is a more filtered version of the train split codeparrot-clean-train of codeparrot-clean. The additional filters aim at detecting configuration and test files, as well as outlier files that are unlikely to help the model learn code. The first three filters are applied with a probability of 0.7:
files with a mention of "test file" or "configuration file" or… See the full description on the dataset page: https://huggingface.co/datasets/codeparrot/codeparrot-train-more-filtering.stack-v3-train
🥞 The Stack v3
What is it?
What is being released
How to download and use it
Dataset statistics
Dataset structure
Dataset creation
Considerations for using the data
Additional information
What is it?
The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context. It is the successor to The Stack v2 and, like its predecessor, is released to make the training… See the full description on the dataset page: https://huggingface.co/datasets/TechnoBaptist/stack-v3-train.the-stack-v2-train-smol-ids-updatedUpdate on The Stack V2 dataset: https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids
All repos from original dataset are parsed with Github API and re-downloaded,
so respective updates are kept, metadata is updated. This took 10+ days to process due to GraphQL limits.
Filtering rules
Removed repos with no update in the last 6 years (no updates since September 2019)
Removed files with a single line
Removed repos with a single file
Removed repos with more than 99%… See the full description on the dataset page: https://huggingface.co/datasets/thepowerfuldeez/the-stack-v2-train-smol-ids-updated.feedback_data_training
Repair replay update — September 15, 2026
The split still contains 161,030 weighted rows, with the same category counts:
Category
Rows
Share
Distinct examples before → after
One-shot
79,970
49.66%
35,197 → 35,197
Regular repairs
60,931
37.84%
40,530 → 48,726
Rollout-derived deep repairs
20,129
12.50%
436 → 1,825
This adds 9,585 distinct checked repair examples while preserving every legacy distinct row and every one-shot row's multiplicity. The new examples… See the full description on the dataset page: https://huggingface.co/datasets/formalmathatepfl/feedback_data_training.python-image-copilot-training-using-import-knowledge-graphs
Python Copilot Image Training using Import Knowledge Graphs
This dataset is a subset of the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details
Each row contains a png file in the dbytes column.
Rows: 216642
Size: 211.2 GB
Data type: png
Format: Knowledge graph using NetworkX with alpaca text box
Schema
The png is in the dbytes column:
{
"dbytes": "binary"… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-image-copilot-training-using-import-knowledge-graphs.bridge_train_10000_15000_augmented
bridge_train_10000_15000_augmented
Overview
Codebase version: v3.0
Robots: google_robot, images, jaco, kinova3, kuka_iiwa, panda, sawyer, ur5e, xarm7
FPS: 5
Episodes: 5,000
Frames: 170,583
Splits:
train: 0:5000
Data Layout
data_path : data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet
video_path: videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4
Features
Feature
dtype
shape
description… See the full description on the dataset page: https://huggingface.co/datasets/oxe-auge/bridge_train_10000_15000_augmented.the-stack-v2-train-smol-ids
The Stack v2
The dataset consists of 4 versions:
bigcode/the-stack-v2: the full "The Stack v2" dataset
bigcode/the-stack-v2-dedup: based on the bigcode/the-stack-v2 but further near-deduplicated
bigcode/the-stack-v2-train-full-ids: based on the bigcode/the-stack-v2-dedup dataset but further filtered with heuristics and spanning 600+ programming languages. The data is grouped into repositories.
bigcode/the-stack-v2-train-smol-ids: based on the bigcode/the-stack-v2-dedup… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/the-stack-v2-train-smol-ids.Amazon-Fashion-Training-Data-2023
Amazon Reviews 2023
Please also visit amazon-reviews-2023.github.io/ for more details, loading scripts, and preprocessed benchmark files.
[April 7, 2024] We add two useful files:
all_categories.txt: 34 lines (33 categories + "Unknown"), each line contains a category name.
asin2category.json: A mapping between parent_asin (item ID) to its corresponding category name.
This is a large-scale Amazon Reviews dataset, collected in 2023 by McAuley Lab, and it includes rich features… See the full description on the dataset page: https://huggingface.co/datasets/Pandeymp29/Amazon-Fashion-Training-Data-2023.the-stack-v2-train-full-ids
The Stack v2
The dataset consists of 4 versions:
bigcode/the-stack-v2: the full "The Stack v2" dataset
bigcode/the-stack-v2-dedup: based on the bigcode/the-stack-v2 but further near-deduplicated
bigcode/the-stack-v2-train-full-ids: based on the bigcode/the-stack-v2-dedup dataset but further filtered with heuristics and spanning 600+ programming languages. The data is grouped into repositories. <-- you are here
bigcode/the-stack-v2-train-smol-ids: based on the… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/the-stack-v2-train-full-ids.OMol25_train_neutral
Cite this dataset Levine, D. S., Shuaibi, M., Spotte-Smith, E. W. C., Taylor, M. G., Hasyim, M. R., Michel, K., Batatia, I., Csányi, G., Dzamba, M., Eastman, P., Frey, N. C., Fu, X., Gharakhanyan, V., Krishnapriyan, A. S., Rackers, J. A., Raja, S., Rizvi, A., Rosen, A. S., Ulissi, Z., Vargas, S., Zitnick, C. L., Blau, S. M., and Wood, B. M. OMol25 train neutral. ColabFit, 2025. https://doi.org/10.60732/3c2ddc75
This dataset has been curated and formatted for the ColabFit… See the full description on the dataset page: https://huggingface.co/datasets/colabfit/OMol25_train_neutral.codeparrot-train-v2-near-dedup-safe
Dataset Card for "codeparrot-train-v2-near-dedup-safe"
More Information needed
abc130k_v3_train
ABC-130k — 256² full-frame re-conversion (train split)
Full-resolution re-conversion of the ABC-130k dual-YAM bimanual manipulation corpus
(XDOF/ABC-130k raw recordings,
Apache-2.0) in LeRobot v3.0 format.
This split: 129,032 episodes · 381,742,382 frames · 197 tasks · 30 fps.
Validation split: huzheyuan/abc130k_v3_val
(1,592 episodes / 189 tasks, task strings aligned to this split).
Why another port
The existing LeRobot port (lerobot/abc_130k_v3_train)… See the full description on the dataset page: https://huggingface.co/datasets/huzheyuan/abc130k_v3_train.