datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SWE-Bench-ProMax
🚀 SWE-Bench-ProMax
A Contamination-Resistant, Multi-Language Benchmark for Repository-Level Issue Resolution
📄 Paper ·
📊 Dataset ·
⚙️ Evaluation ·
📬 Contact
SWE-Bench-ProMax is a multi-language benchmark for repository-level software issue resolution, accepted to COLM 2026. All task instances are collected from real-world GitHub issues created after 2025, substantially reducing the risk of training-data contamination for contemporary language models. Each instance provides a task… See the full description on the dataset page: https://huggingface.co/datasets/swe-bench-promax/SWE-Bench-ProMax.SWEBench-Pro-Verified
SWE-Bench Pro Verified: Anti-hacking & Task refinement
SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging
repository-level tasks. However, our analysis work show that its evaluation is undermined by two sources of
unreliability: reward hacking, enabled by leakage of gold solutions or hidden evaluation information, and
task quality issues, including misleading problem statements and improperly scoped tests. These issues can… See the full description on the dataset page: https://huggingface.co/datasets/opencompass/SWEBench-Pro-Verified.swebench-localisation
Finding the file: localisation on SWE-bench Verified
Given a GitHub issue, which file do you have to change? This is the retrieval step every coding
agent performs before it writes a patch, and none of the leaderboards score it separately. SWE-bench's five
leaderboards all score % Resolved, which folds localisation and patch-writing into one
number.
This bundle is that step measured on its own, on all 500 instances of SWE-bench Verified, with a
floor. The write-up is
Finding… See the full description on the dataset page: https://huggingface.co/datasets/RiverRider/swebench-localisation.qwen36-kquant-offload-mtp-swebench-lite100-results
Qwen3.6 K-Quant Offload MTP SWE-bench Lite 100 Results
This dataset contains the complete 5-model x 100-prompt runtime benchmark artifacts plus a detailed statistical analysis layer.
Primary conclusion: hot30/cold30 was the best decode-throughput run, while Q4_K_M had the best total wall clock. The ATX hot30/cold30 quantization significantly outperformed both Q4_K_M and Q3_K_XL on paired decode throughput, but Q4_K_M remains the elapsed-time control.
The ATX/K3 hot10, hot20, and… See the full description on the dataset page: https://huggingface.co/datasets/jakeatx/qwen36-kquant-offload-mtp-swebench-lite100-results.swe-bench-lite-agent-traces-v14
AgentBRANE SWE-bench Lite Agent Traces v14
This release contains the 1,890 harness-native agent traces selected by the sealed SWE-bench Lite v14 publication record (1,379/1,890 resolved, 73.0%). It includes Claude Code, Codex, and Pi sessions across seven models and three replicates. No internal research notes are included.
Load the observation table:
from datasets import load_dataset
traces = load_dataset("melissapan/swe-bench-lite-agent-traces-v14", split="train")
Each row… See the full description on the dataset page: https://huggingface.co/datasets/melissapan/swe-bench-lite-agent-traces-v14.swebench-cc-traces-rawswebench-cc-traces-anonswe_bench_qiskit_testswebench-verified-deepseek-v4-flash-failure-analysis
SWE-bench Verified runs & failure analysis — DeepSeek-V4-flash (local) × mini-swe-agent
Per-instance analysis of SWE-bench Verified runs of a locally-served DeepSeek-V4-flash model
driven by mini-swe-agent, graded with the official
SWE-bench harness. Each instance carries the full agent trajectory, a readable transcript, the
submitted patch, the harness test output, deterministic metrics, and a hand-verified qualitative
root-cause diagnosis.
Current numbers (resolve rates… See the full description on the dataset page: https://huggingface.co/datasets/daaain/swebench-verified-deepseek-v4-flash-failure-analysis.swebench-minimax-traces
swebench-minimax-traces
Complete HTTP-level agentic traces from running swebench benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 500
Multi-turn sessions (2+ LLM calls): 357
Total records: 22424
Total LLM requests: 11249
Format
Raw JSONL traces from the instrumented proxy.… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-minimax-traces.swebench-sonnet-traces
swebench-sonnet-traces
Complete HTTP-level agentic traces from running swebench_sonnet benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 122
Multi-turn sessions (2+ LLM calls): 69
Total records: 2748
Total LLM requests: 1374
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-sonnet-traces.swebench-deepseek-traces
swebench-deepseek-traces
Complete HTTP-level agentic traces from running swebench_deepseek benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 50
Multi-turn sessions (2+ LLM calls): 18
Total records: 542
Total LLM requests: 271
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-deepseek-traces.swe-bench-verified-raw-traces-qwen3-coder
SWE-bench Verified raw mini-SWE-agent traces
Raw mini-SWE-agent trajectories from 20250802_mini-v1.0.0_qwen3-coder-480b-a35b-instruct for SWE-bench Verified.
The raw/easy split uses exactly the 194 instance IDs from
parsaidp/SWE-bench_Verified_easy. That public dataset contains SWE-bench
Verified questions and Kimi-generated answers; this dataset uses only its
instance IDs. The trajectory contents here are local mini-SWE-agent/Qwen traces.
Files
data/full.jsonl:… See the full description on the dataset page: https://huggingface.co/datasets/nikitamounier/swe-bench-verified-raw-traces-qwen3-coder.swebench-deepseek-traces-3
swebench-deepseek-traces-3
Complete HTTP-level agentic traces from running swebench_deepseek benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 46
Multi-turn sessions (2+ LLM calls): 26
Total records: 2154
Total LLM requests: 1077
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-deepseek-traces-3.swebench-deepseek-traces-2
swebench-deepseek-traces-2
Complete HTTP-level agentic traces from running swebench_deepseek benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 37
Multi-turn sessions (2+ LLM calls): 33
Total records: 2040
Total LLM requests: 1020
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-deepseek-traces-2.swe-bench-verified-raw-traces-qwen3-coder
SWE-bench Verified raw mini-SWE-agent traces
Raw mini-SWE-agent trajectories from 20250802_mini-v1.0.0_qwen3-coder-480b-a35b-instruct for SWE-bench Verified.
The raw/easy split uses exactly the 194 instance IDs from parsaidp/SWE-bench_Verified_easy. That
public dataset contains SWE-bench Verified questions and Kimi-generated answers; this
dataset uses only its instance IDs. The trajectory contents here are local
mini-SWE-agent/Qwen traces.
Files
data/full.jsonl:… See the full description on the dataset page: https://huggingface.co/datasets/parsaidp/swe-bench-verified-raw-traces-qwen3-coder.swebench-minimax-traces-2
swebench-minimax-traces-2
Complete HTTP-level agentic traces from running swebench benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 250
Multi-turn sessions (2+ LLM calls): 210
Total records: 15198
Total LLM requests: 7623
Format
Raw JSONL traces from the instrumented proxy.… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-minimax-traces-2.swebench-sonnet-traces-2
swebench-sonnet-traces-2
Complete HTTP-level agentic traces from running swebench_sonnet benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 108
Multi-turn sessions (2+ LLM calls): 45
Total records: 2552
Total LLM requests: 1276
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-sonnet-traces-2.swe_doc_gen_locate_swebench_1500
SWE-Doc-Gen-Locate Dataset (SWE-Bench 1500 entries)
A dataset for evaluating an agent's ability to locate a target Python function/class based on its functionality description and add a docstring.
Task Description
Given: A Python repository and a description of a function/class (NO name, NO file path)
Agent must:
Search the codebase to find where the target function/class is defined
Read the implementation to understand its behavior
Generate and add an appropriate… See the full description on the dataset page: https://huggingface.co/datasets/synthetic-code-training/swe_doc_gen_locate_swebench_1500.bullet-swebench-verified
Bullet on SWE-bench Verified — 479/500 = 95.8%
Results for the Bullet coding agent on all 500 instances of
SWE-bench Verified, graded by the official swebench.harness.run_evaluation
scorer. Every instance was attempted and graded; there are no empty patches.
479 / 500 resolved = 95.8%
Run with the Bullet harness on gpt-5.6-sol at high reasoning effort, one attempt per instance.
By repository
repository
resolved
django
223/231
96.5%
sympy
73/75
97.3%… See the full description on the dataset page: https://huggingface.co/datasets/daviddata1/bullet-swebench-verified.blinker-swe-bench