datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SWE-rebench-leaderboard
Dataset Summary
❗❗❗ Please use Harbour Hub for the July 2026 evaluation split:https://hub.harborframework.com/datasets/ibragim-badertdinov/swe-rebench-07-2026/latest
SWE-rebench-leaderboard is a continuously updated, curated subset of the full SWE-rebench corpus, tailored for benchmarking software engineering agents on real-world tasks.
These tasks are used in the SWE-rebench leaderboard. For more details on the benchmark methodology and data collection process, please refer to… See the full description on the dataset page: https://huggingface.co/datasets/nebius/SWE-rebench-leaderboard.SWE-PolyBench
SWE-PolyBench
SWE-PolyBench is a multi language repo level software engineering benchmark. Currently it includes 4 languages: Python, Java, Javascript, and Typescript. The number of instances in each language is:
Javascript: 1017
Typescript: 729
Python: 199
Java: 165
Datasets
There are total three datasets available under SWE-PolyBench. AmazonScience/SWE-PolyBench is the full dataset, AmazonScience/SWE-PolyBench_500 is the stratified sampled dataset with 500 instances and… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/SWE-PolyBench.SWE-chat
SWE-chat: Coding Agent Interactions From Real Users in the Wild
📄 Paper: arxiv.org/abs/2604.20779
🌐 Website: swe-chat.com
Dataset Summary
SWE-chat captures real-world AI coding sessions from developers using AI coding assistants (Claude Code, Codex, Gemini CLI, and others via the Entire.io CLI). Each session includes the full conversation transcript, tool calls, thinking traces, code changes, and attribution of human vs. agent-authored code.
Dataset Size… See the full description on the dataset page: https://huggingface.co/datasets/SALT-NLP/SWE-chat.Multi-SWE-RL-Verified
Multi-SWE-RL-Verified
Gold-patch-validated subset of
PrimeIntellect/Multi-SWE-RL-Reupload
(ByteDance's Multi-SWE-RL): 2,232 / 4,703 rows across
C, Go, Java, JavaScript, Rust, and TypeScript that produce a clean reward signal end-to-end.
Default dataset of the multiswe_v1 taskset.
Changes vs upstream
Starting from the 4,703-row re-upload:
C++ dropped wholesale — 0/449 rows passed gold-patch validation in pass 1; the images are
broken for scoring, not merely… See the full description on the dataset page: https://huggingface.co/datasets/PrimeIntellect/Multi-SWE-RL-Verified.SWE-rebench-openhands-trajectories
Dataset Summary
SWE-rebench-OpenHands-Trajectories is a dataset of multi-turn agent trajectories for software engineering tasks, collected
using Qwen/Qwen3-Coder-480B-A35B-Instruct with OpenHands (v0.54.0) agent scaffolding.
This dataset captures complete agent execution traces as they attempt to resolve real GitHub issues from
nebius/SWE-rebench.
Each trajectory contains the agent's step-by-step reasoning, actions, and environmental observations.
Metric… See the full description on the dataset page: https://huggingface.co/datasets/nebius/SWE-rebench-openhands-trajectories.SWE-bench-cSWE-bench-C is a benchmark for evaluating AI models for Software Engineering Tasks in C language. The dataset is similar to SWE-bench but for the C domain. Issues and pull requests from 3 popular GitHub C repositories facebook/zstd, jqlang/jq & redis/redis were scrapped. Filtering PRs from these repos resulted in 178PRs.
epoch_ai_swebench_verified
Epoch AI SWE-bench Verified Traces
Complete public trace archives and an analysis-ready Parquet conversion of Epoch AI's SWE-bench Verified evaluations.
Contents
34 published evaluation runs covering 16,456 traces (484 SWE-bench instances per run).
data/: loadable Parquet data, one exact trace per row.
original/: the byte-identical .eval archives published by Epoch AI.
run_metadata/: non-sample files from each .eval archive (header.json, summaries, reductions… See the full description on the dataset page: https://huggingface.co/datasets/naderalfares/epoch_ai_swebench_verified.sweden_100K_difficultSWE-PolyBench_Verified
SWE-PolyBench
SWE-PolyBench is a multi language repo level software engineering benchmark. Currently it includes 4 languages: Python, Java, Javascript, and Typescript. The number of instances in the verified split is:
Javascript: 100
Typescript: 100
Python: 113
Java: 69
Datasets
There are total three datasets available under SWE-PolyBench. AmazonScience/SWE-PolyBench is the full dataset, AmazonScience/SWE-PolyBench_500 is the stratified sampled dataset with 500 instances… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/SWE-PolyBench_Verified.swe_jsts_initprlarge100swe-marathon
SWE Marathon: Ultra Long-Horizon Software Engineering Tasks
20 ultra long-horizon software-engineering tasks designed to challenge frontier coding agents. Each task ships with a containerized environment, a precise instruction, comprehensive tests, and a reference oracle solution. All tasks pass NOP-baseline / Oracle-fix validation.
Homepage: https://github.com/abundant-ai/swe-marathon
License: Apache 2.0
Format: Harbor task format (task.toml + instruction.md + environment/ +… See the full description on the dataset page: https://huggingface.co/datasets/rdesai2/swe-marathon.SWE-Bench-ProMax
🚀 SWE-Bench-ProMax
A Contamination-Resistant, Multi-Language Benchmark for Repository-Level Issue Resolution
📄 Paper ·
📊 Dataset ·
⚙️ Evaluation ·
📬 Contact
SWE-Bench-ProMax is a multi-language benchmark for repository-level software issue resolution, accepted to COLM 2026. All task instances are collected from real-world GitHub issues created after 2025, substantially reducing the risk of training-data contamination for contemporary language models. Each instance provides a task… See the full description on the dataset page: https://huggingface.co/datasets/swe-bench-promax/SWE-Bench-ProMax.so100_sweeper_shoesThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.0",
"robot_type": "so100",
"total_episodes": 774,
"total_frames": 2145169,
"total_tasks": 1,
"total_videos": 3096,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 29,
"splits": {
"train": "0:774"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/Yotofu/so100_sweeper_shoes.SWE-PolyBench_500
SWE-PolyBench
SWE-PolyBench is a multi language repo level software engineering benchmark. Currently it includes 4 languages: Python, Java, Javascript, and Typescript. The number of instances in each language is:
Javascript: 1017
Typescript: 729
Python: 199
Java: 165
Datasets
There are total three datasets available under SWE-PolyBench. AmazonScience/SWE-PolyBench is the full dataset, AmazonScience/SWE-PolyBench_500 is the stratified sampled dataset with 500 instances and… See the full description on the dataset page: https://huggingface.co/datasets/AmazonScience/SWE-PolyBench_500.SWE-Dev-train📝 Paper | 🌐 Github
🤗 SWE-Dev-7B (Qwen-2.5-Coder-7B-Instruct)
🤗 SWE-Dev-9B (GLM-4-9B-Chat)
🤗 SWE-Dev-32B (Qwen-2.5-Coder-32B-Instruct)
🤗 SWE-Dev-train (Training Data)
🚀 SWE-Dev, an open-source Agent for Software Engineering tasks! This repository contains the SWE-Dev-32B model as presented in the paper SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling.
💡 We develop a comprehensive pipeline for creating developer-oriented datasets from GitHub… See the full description on the dataset page: https://huggingface.co/datasets/zai-org/SWE-Dev-train.SWEBench-Pro-Verified
SWE-Bench Pro Verified: Anti-hacking & Task refinement
SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging
repository-level tasks. However, our analysis work show that its evaluation is undermined by two sources of
unreliability: reward hacking, enabled by leakage of gold solutions or hidden evaluation information, and
task quality issues, including misleading problem statements and improperly scoped tests. These issues can… See the full description on the dataset page: https://huggingface.co/datasets/opencompass/SWEBench-Pro-Verified.dvla-can2k-futee-sweep
can2k — action-label lookahead sweep
Four LeRobot datasets built from the same 2,005 MuJoCo/robosuite demonstrations. The images,
states and episode boundaries are identical across all four; only the action label differs, by
the size of the future-EE relabel window (make_lerobot --futee_offset).
variant
offset
lookahead @250 fps
frames
leading hold trimmed
futee0
0
0 ms
1,695,122
55.0
futee20
20
80 ms
1,705,362
49.9
futee40
40
160 ms
1,745,478
29.9
futee60
60… See the full description on the dataset page: https://huggingface.co/datasets/mickeykang/dvla-can2k-futee-sweep.SWE-ZERO-96K-trajectories
SWE-ZERO 1B Trajectories
96,237 execution-free agentic rollouts across 32,079 pull requests in 20 programming languages from SWE-rebench V2.
Overview
Each rollout is a multi-turn trajectory where a small code model (ricdomolm/mini-coder-1.7b, Qwen3-1.7B fine-tuned on 400k mini-swe-agent trajectories) attempts to fix a real GitHub issue using only bash commands (cat, grep, sed, find, etc.) without access to language interpreters or test execution.
Rollouts: 96,237 (3 per… See the full description on the dataset page: https://huggingface.co/datasets/AlienKevin/SWE-ZERO-96K-trajectories.swepro-luna-matched-pair
SWE-bench Pro, matched pair: Ouroboros vs Codex CLI on one model
Status: Self-reported matched-pair study. Both harnesses used the same
model, task set and evaluator. The strict result is a statistical tie.
Start here
Strict result
Ouroboros 58.2%, Codex CLI 59.4%, McNemar p = 0.40
Model
openai/gpt-5.6-luna for both arms
Filter
655 paired tasks after the same reference-leak filter was applied to both arms
Exact evidence
6228037, manifest.csv… See the full description on the dataset page: https://huggingface.co/datasets/razzant/swepro-luna-matched-pair.SWE-Rebench-Tasks-Clean
SWE-Rebench-Tasks-Clean
1,317 verified-solvable, contamination-controlled software-engineering tasks for terminal-agent RL training.
Adapted from nebius/SWE-rebench-V2 (real GitHub issue → PR tasks with executable test contracts) into the TerminalWorld task format. Companion dataset to Fzz1/SWE-Smith-Seeds-Clean, same layout.
Every task is a directory containing:
file
content
instruction.md
the issue text the agent sees (plus linked issue discussion where available)… See the full description on the dataset page: https://huggingface.co/datasets/Fzz1/SWE-Rebench-Tasks-Clean.deep-swe
DeepSWE
DeepSWE is a benchmark for measuring frontier coding agents on original, long-horizon software engineering tasks drawn from active open-source repositories. The benchmark includes 113 tasks across TypeScript, Go, Python, JavaScript, and Rust, with isolated environments and program-based verifiers.
Task format
DeepSWE tasks use the Harbor task format:
task.toml Metadata: repository, base commit, language, prebuilt image, resource limits… See the full description on the dataset page: https://huggingface.co/datasets/datacurve/deep-swe.Nemotron-RL-Agentic-SWE-Pivot-v1
Dataset Description:
The SWE-RL dataset provides GitHub issues for training and validating real-world software engineering agents using the OpenHands environment in NeMo Gym. The dataset is a refactored version of the SWE-Gym and R2E-Gym datasets to support the NeMo Gym input format.
This dataset is released as part of NVIDIA NeMo Gym, a framework for building reinforcement learning environments to train large language models. NeMo Gym contains a growing collection of training… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-SWE-Pivot-v1.swe-prbench
SWE-PRBench
Benchmarking AI Code Review Quality Against Human Pull Request Feedback
Blog: Read the blog
GitHub Repository: View the code
arXiv Paper: View the paper
Overview
SWE-PRBench is a benchmark of 350 pull requests with human-annotated
ground truth for evaluating whether LLMs can identify the same issues
that real human reviewers flag in production code.
Existing benchmarks like SWE-Bench measure whether models can produce
correct code. SWE-PRBench… See the full description on the dataset page: https://huggingface.co/datasets/foundry-ai/swe-prbench.medical-symptom-triage-conversationalswebench-localisation
Finding the file: localisation on SWE-bench Verified
Given a GitHub issue, which file do you have to change? This is the retrieval step every coding
agent performs before it writes a patch, and none of the leaderboards score it separately. SWE-bench's five
leaderboards all score % Resolved, which folds localisation and patch-writing into one
number.
This bundle is that step measured on its own, on all 500 instances of SWE-bench Verified, with a
floor. The write-up is
Finding… See the full description on the dataset page: https://huggingface.co/datasets/RiverRider/swebench-localisation.deep-swe-1-1-materialized
DeepSWE 1.1 — materialized
A tabular materialization of DeepSWE
v1.1 — Datacurve's 113-task benchmark for coding agents — repackaged from
datacurve-ai/deep-swe into one
parquet row per task. This is a third-party repack for tooling convenience,
not an official Datacurve release.
Source commit: see manifest.json (source_commit) — every file is
carried over unmodified into columns.
Integrity: manifest.json records the parquet's sha256 and a per-task
content hash (sha256 over each… See the full description on the dataset page: https://huggingface.co/datasets/luolc/deep-swe-1-1-materialized.SWE-Review-Chat
SWE-Review-Chat: A Dataset of Code Review Conversations and Human-AI Collaboration in Agentic Code Review
Paper: https://arxiv.org/abs/2607.13196
GitHub: https://github.com/suzhenxzhong/SWE-Review-Chat
SWE-Review-Chat is a large-scale dataset of real-world code review conversations from pull requests of 207 popular GitHub projects, spanning the transition from human-centric to LLM-assisted and agentic code review by AI agents.
📊 Dataset Overview
Field… See the full description on the dataset page: https://huggingface.co/datasets/Suzhen/SWE-Review-Chat.qwen36-kquant-offload-mtp-swebench-lite100-results
Qwen3.6 K-Quant Offload MTP SWE-bench Lite 100 Results
This dataset contains the complete 5-model x 100-prompt runtime benchmark artifacts plus a detailed statistical analysis layer.
Primary conclusion: hot30/cold30 was the best decode-throughput run, while Q4_K_M had the best total wall clock. The ATX hot30/cold30 quantization significantly outperformed both Q4_K_M and Q3_K_XL on paired decode throughput, but Q4_K_M remains the elapsed-time control.
The ATX/K3 hot10, hot20, and… See the full description on the dataset page: https://huggingface.co/datasets/jakeatx/qwen36-kquant-offload-mtp-swebench-lite100-results.SWE-MERA
SWE-MERA
Continuously updated SWE-MERA dataset
SWE-MERA splits:
dev: for testing (10 samples)
lite: presented at the leaderboard here (750 samples)
full: continuously updated to collect more data (2738 samples)
Load dataset
from datasets import load_dataset
ds = load_dataset("MERA-evaluation/SWE-MERA", split='dev')
Evaluation
Description
The main tool to validate tasks is repotest (available at PyPI or GitHub)
data.jsonl -… See the full description on the dataset page: https://huggingface.co/datasets/MERA-evaluation/SWE-MERA.harbor-swesmith-rl-artifacts
Harbor SWE-Smith 强化学习数据产物
本数据集是 Harbor Qwen 工具调用代码智能体强化学习项目使用的冻结任务集，服务于 GRPO、原生价值模型/GAE PPO、训练过程诊断和统一协议评测。
项目已于 2026 年 8 月 30 日完成 P0 评测并进入阶段性归档。本数据集用于保留实验所依赖的数据切分、任务执行文件和审计信息，不代表新的通用代码能力基准。
数据概况
切分
任务数
训练集
187
验证集
42
测试集
38
合计
267
数据覆盖 89 个上游代码仓库。三个切分之间同时执行任务标识和仓库级隔离检查。
正式数据集名称：
swesmith-curated-grpo-267-v1
冻结切分的语义摘要：
ae5df9a3f4a3fc8af44fac420b36529e283839e1bd3de9daba65d5bcda51447d
该值来自 split-manifest.json 的 sha256 字段，用于标识切分语义，不等同于该文件本身的字节级… See the full description on the dataset page: https://huggingface.co/datasets/keryszhan/harbor-swesmith-rl-artifacts.