datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
swe-bench-dummy-test-datasetSWE-bench-JS
Dataset Name
Private dataset.
Single JSONL file (~9.6MB).
Multi-SWE-bench
SWE-bench-Java: A GitHub Issue Resolving Benchmark for Java
📰 News
[Aug. 27, 2024]:We’ve released the JAVA version of SWE-bench! Check it out on Hugging Face. For more details, see our paper!
📄 Abstract
GitHub issue resolving is a critical task in software engineering, recently gaining significant attention in both industry and academia. Within this task, SWE-bench has been released to evaluate issue resolving capabilities of large language models (LLMs)… See the full description on the dataset page: https://huggingface.co/datasets/Daoguang/Multi-SWE-bench.SWE-Bench-ProMax
🚀 SWE-Bench-ProMax
A Contamination-Resistant, Multi-Language Benchmark for Repository-Level Issue Resolution
📄 Paper ·
📊 Dataset ·
⚙️ Evaluation ·
📬 Contact
SWE-Bench-ProMax is a multi-language benchmark for repository-level software issue resolution, accepted to COLM 2026. All task instances are collected from real-world GitHub issues created after 2025, substantially reducing the risk of training-data contamination for contemporary language models. Each instance provides a task… See the full description on the dataset page: https://huggingface.co/datasets/swe-bench-promax/SWE-Bench-ProMax.SWEBench-Pro-Verified
SWE-Bench Pro Verified: Anti-hacking & Task refinement
SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging
repository-level tasks. However, our analysis work show that its evaluation is undermined by two sources of
unreliability: reward hacking, enabled by leakage of gold solutions or hidden evaluation information, and
task quality issues, including misleading problem statements and improperly scoped tests. These issues can… See the full description on the dataset page: https://huggingface.co/datasets/opencompass/SWEBench-Pro-Verified.SWE-QA-Benchmark
SWE-QA Benchmark
A comprehensive benchmark dataset for Software Engineering Question Answering, containing 720 questions across 15 popular Python repositories.
Dataset Summary
Total Questions: 720
Repositories: 15
Format: JSONL (JSON Lines)
Fields: question, answer
Repository Coverage
Each repository contains 48 questions:
astropy
conan
django
flask
matplotlib
pylint
pytest
reflex
requests
scikit-learn
sphinx
sqlfluff
streamlink
sympy
xarray… See the full description on the dataset page: https://huggingface.co/datasets/swe-qa/SWE-QA-Benchmark.swebench-localisation
Finding the file: localisation on SWE-bench Verified
Given a GitHub issue, which file do you have to change? This is the retrieval step every coding
agent performs before it writes a patch, and none of the leaderboards score it separately. SWE-bench's five
leaderboards all score % Resolved, which folds localisation and patch-writing into one
number.
This bundle is that step measured on its own, on all 500 instances of SWE-bench Verified, with a
floor. The write-up is
Finding… See the full description on the dataset page: https://huggingface.co/datasets/RiverRider/swebench-localisation.qwen36-kquant-offload-mtp-swebench-lite100-results
Qwen3.6 K-Quant Offload MTP SWE-bench Lite 100 Results
This dataset contains the complete 5-model x 100-prompt runtime benchmark artifacts plus a detailed statistical analysis layer.
Primary conclusion: hot30/cold30 was the best decode-throughput run, while Q4_K_M had the best total wall clock. The ATX hot30/cold30 quantization significantly outperformed both Q4_K_M and Q3_K_XL on paired decode throughput, but Q4_K_M remains the elapsed-time control.
The ATX/K3 hot10, hot20, and… See the full description on the dataset page: https://huggingface.co/datasets/jakeatx/qwen36-kquant-offload-mtp-swebench-lite100-results.SWE-QA-Pro-Bench
SWE-QA-Pro Bench (A Repository-level QA Benchmark Built from Diverse Long-tail Repositories)
💻 GitHub | 📖 Paper | 🤗 SWE-QA-Pro
📢 News
🚀 [2026-5-19] The evaluation code is released on GitHub.
🔥 [2026-3-23] SWE-QA-Pro Bench is publicly released! The model and code will be released soon.
Introduction
SWE-QA-Pro Bench is a repository-level question answering dataset designed to evaluate whether models can perform grounded, agentic reasoning… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/SWE-QA-Pro-Bench.swe-bench-mini
SWE-bench-mini
34 self-contained bug-fix tasks in the SWE-bench format — a small repository snapshot
carrying a defect, a test that fails because of it, and a gold patch that fixes it (difficulty
mix: 12 easy / 19 medium / 3 hard, author estimate). Built for the swe_bench_mini agent and the
make demo-swe-mini evaluator in
adk-agent-playground, to demonstrate
the framework's range on code-modification and to exercise the CaMeL filesystem-capability gate.
A second harder config… See the full description on the dataset page: https://huggingface.co/datasets/barissozudogru/swe-bench-mini.codex_swebenchpro_tracesThis is a dataset generated by real swebenchpro agentic workload trace + codex agent.
1. Eval Result Summary
Metric
Value
Total trials
731
Successful trials
610
Failed trials
120
No data (skipped)
1
Passed
329
Pass rate (of successful)
53.9%
Per-Repo Breakdown
Repo
Total
Success
Failed
Passed
Pass%
ansible/ansible
96
93
3
60
65%
internetarchive/openli
91
88
3
52
59%
flipt-io/flipt85
82
3
26
32%
qutebrowser/qutebrowse
79
78
1… See the full description on the dataset page: https://huggingface.co/datasets/Inferact/codex_swebenchpro_traces.SWE-bench_oracle_scikitSWE-bench-plus
SWE-bench-Plus: Test Enhancer
SWE-bench-Plus is a coverage-guided test generation and evaluation layer built on top of the official SWE-bench harness. It automates iterative LLM-based test generation, avoids duplicates, targets uncovered code paths, and stops when coverage plateaus. It is designed for high-throughput, resume-friendly batch runs with robust logging and fault tolerance.
Key Features
Coverage-guided generation: After each iteration, the harness measures… See the full description on the dataset page: https://huggingface.co/datasets/hrtxsny/SWE-bench-plus.SWE-Skills-BenchDataset Summary
SWE-Skills-Bench is a benchmark dataset for evaluating whether injected skill documents — structured packages of procedural knowledge — measurably improve LLM agent performance on real-world software engineering tasks.
The dataset contains 49 skills spanning 565 task instances across six software engineering domains (Deployment & DevOps, Analytics & Monitoring, API Development, Data Science & ML, Security & Testing, and Developer Tools). Each skill is grounded in an authentic… See the full description on the dataset page: https://huggingface.co/datasets/GeniusHTX/SWE-Skills-Bench.SWE-Explore-Bench
SWE-Explore-Bench
SWE-Explore-Bench is the dataset for SWE-Explore: Benchmarking How Coding Agents Explore Repositories.
Citation
If you use SWE-Explore-Bench, please cite:
@misc{zhang2026sweexplore,
title = {{SWE-Explore}: Benchmarking How Coding Agents Explore Repositories},
author = {Shaoqiu Zhang and Yuhang Wang and Jialiang Liang and Yuling Shi and Wenhao Zeng and Maoquan Wang and Shilin He and Ningyuan Xu and Siyu Ye and Kai Cai and Xiaodong Gu},
year… See the full description on the dataset page: https://huggingface.co/datasets/SWE-Explore-Bench/SWE-Explore-Bench.go-swe-bench-v0
go_swe_bench v0 — real Go bug fixes, verified by the Go toolchain
246 tasks from 79 real Go repositories. Each task is a bug-fix commit whose co-committed test is red on the
parent and green on the fix. No LLM anywhere in the build.
Mined on 2026-09-19 from the GuildLM Go mining pipeline by inverting the filter that had thrown the tests
away (the pipeline was built for SFT data; a benchmark needs the opposite). Every task was verified twice
with go test: green at the commit (≥ 1… See the full description on the dataset page: https://huggingface.co/datasets/guildlm/go-swe-bench-v0.SWEUniverse-Repaired-Indist-full-not-SWE-bench-pro-matched
VmaxRL/SWEUniverse-Repaired-Indist-full-not-SWE-bench-pro-matched
This dataset contains a 350-row subset selected from the Indist SWEUniverse training rows.
Selection policy: three-way repo overlap with Bugpilot and LM-Modify, deduped by repo plus introduction patch, then balanced round-robin across overlapping repos.
Rows: 350
Selected repos: 19
Deduped overlap capacity: 468
Source dataset: VmaxRL/SWEUniverse-Repaired-Indist-full-not-SWE-bench-pro-matched
nemotron-nano-30b-miniswe-swebench-verified
Nemotron Nano 30B + mini-swe-agent SWE-bench Verified Trajectories
Agent trajectories from running NVIDIA Nemotron 3 Nano 30B A3B (MoE, 8B active params) on SWE-bench Verified using mini-swe-agent.
⚠️ Incomplete Run
This benchmark was terminated early due to poor performance. The model struggled with the agentic coding task.
Model Information
Attribute
Value
Model
NVIDIA Nemotron 3 Nano 30B A3B
Architecture
MoE (30B total, 8B active)
Serving
vLLM… See the full description on the dataset page: https://huggingface.co/datasets/pankajmathur/nemotron-nano-30b-miniswe-swebench-verified.swebenchproLego-RL-SWE-Bench-Verified
Lego-RL-SWE-Bench-Verified
The 500 SWE-bench Verified instances as ready-to-run harbor RL environments —
the exact evaluation set behind every SWE-bench Verified number in
LEGO-RL, packaged the same way as the training
set Lego-X/Lego-RL-2699 so
one trainer reads both.
Two parallel views of the same 500 instances:
View
Path
What it is
Official SWE-bench records
swebench_verified_official_500/
The upstream princeton-nlp/SWE-bench_Verified rows, verbatim
Harbor RL… See the full description on the dataset page: https://huggingface.co/datasets/Lego-X/Lego-RL-SWE-Bench-Verified.SWEBench-FileTree
SWEBench-FileTree
Path-only version of embedding-benchmark/SWEbenchCodeRetrieval.
Source code is stripped; only file paths remain. The task: given a GitHub issue
and a repository snapshot represented by its file tree alone, return the file
that needs to be modified.
This is a routing benchmark, not a code retrieval benchmark. It measures how
far path tokens, filenames, extensions, and directory structure get you before
anyone reads a line of source.
Task
Input… See the full description on the dataset page: https://huggingface.co/datasets/AmuroEita/SWEBench-FileTree.swe-bench-opus-logs
Claude 3 inference SWE-Bench results
Contains prompting responses from SWE-bench on these 2 settings:
Oracle retrieval
BM25 retrieval
Each of the subsets contains an additional log_last_line attributes which is the last line from log files generated during evaluation step.
Results:
Model
BM25 Retrieval Resolved (%)
Oracle Retrieval Resolved (%)
GPT-4*
0
1.74
Claude-2
1.96
4.80
Claude-3 Opus (20240229)
3.24
6.42
Claude-2 and GPT-4 results from SWE-bench… See the full description on the dataset page: https://huggingface.co/datasets/syntaxsynth/swe-bench-opus-logs.swe-bench-lite-agent-traces-v14
AgentBRANE SWE-bench Lite Agent Traces v14
This release contains the 1,890 harness-native agent traces selected by the sealed SWE-bench Lite v14 publication record (1,379/1,890 resolved, 73.0%). It includes Claude Code, Codex, and Pi sessions across seven models and three replicates. No internal research notes are included.
Load the observation table:
from datasets import load_dataset
traces = load_dataset("melissapan/swe-bench-lite-agent-traces-v14", split="train")
Each row… See the full description on the dataset page: https://huggingface.co/datasets/melissapan/swe-bench-lite-agent-traces-v14.swebench-cc-traces-rawfastapi-swe-bench-tasks
FastAPI SWE-Bench Tasks
5 SWE-Bench-style evaluation instances for tiangolo/fastapi at commit 627c10a2.
Instances
ID
Title
tiangolo__fastapi-90001
Duplicate on_startup/on_shutdown handlers when same router included twice
tiangolo__fastapi-90002
_endpoint_context_cache stale data and memory leak via id() reuse
tiangolo__fastapi-90003
strict_content_type not overridable at include_router call site
tiangolo__fastapi-90004
_extract_endpoint_context swallows… See the full description on the dataset page: https://huggingface.co/datasets/mokshjain/fastapi-swe-bench-tasks.Devin-SWE-bench-outputfastapi-swebench-v2
FastAPI SWE-bench Tasks
5 SWE-bench-style bug-fix tasks mined from tiangolo/fastapi.
Each instance includes:
instance_id, repo, pull_number, base_commit, version
patch — gold source-only fix
test_patch — gold test changes
problem_statement — issue title + body
FAIL_TO_PASS — tests that must go from failing to passing
PASS_TO_PASS — tests that must remain passing
image_tag — pre-built Docker image (mokshjn00/fastapi-swebench:<instance_id>, linux/amd64)
Instances… See the full description on the dataset page: https://huggingface.co/datasets/mokshjain/fastapi-swebench-v2.swebench-cc-traces-anonswe_bench_qiskit_testAweAgent-Meta-SWE-Bench-Pro
AweAgent-Meta-SWE-Bench-Pro
This dataset provides the metadata used by AweAgent to run the SWE-Bench-Pro evaluation.
If you are looking for the underlying benchmark itself (task design, repositories, test suites), please refer to the original project: scaleapi/SWE-bench_Pro-os and the accompanying paper SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? (arXiv:2509.16941).
Files
swe_bench_pro_aweagent.jsonl — one JSON object per… See the full description on the dataset page: https://huggingface.co/datasets/AweAI-Team/AweAgent-Meta-SWE-Bench-Pro.