datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
swe_jsts_initprlarge100swe-marathon
SWE Marathon: Ultra Long-Horizon Software Engineering Tasks
20 ultra long-horizon software-engineering tasks designed to challenge frontier coding agents. Each task ships with a containerized environment, a precise instruction, comprehensive tests, and a reference oracle solution. All tasks pass NOP-baseline / Oracle-fix validation.
Homepage: https://github.com/abundant-ai/swe-marathon
License: Apache 2.0
Format: Harbor task format (task.toml + instruction.md + environment/ +… See the full description on the dataset page: https://huggingface.co/datasets/rdesai2/swe-marathon.SWE-Bench-ProMax
🚀 SWE-Bench-ProMax
A Contamination-Resistant, Multi-Language Benchmark for Repository-Level Issue Resolution
📄 Paper ·
📊 Dataset ·
⚙️ Evaluation ·
📬 Contact
SWE-Bench-ProMax is a multi-language benchmark for repository-level software issue resolution, accepted to COLM 2026. All task instances are collected from real-world GitHub issues created after 2025, substantially reducing the risk of training-data contamination for contemporary language models. Each instance provides a task… See the full description on the dataset page: https://huggingface.co/datasets/swe-bench-promax/SWE-Bench-ProMax.SWEBench-Pro-Verified
SWE-Bench Pro Verified: Anti-hacking & Task refinement
SWE-Bench Pro has emerged as a standard benchmark for evaluating software engineering agents on challenging
repository-level tasks. However, our analysis work show that its evaluation is undermined by two sources of
unreliability: reward hacking, enabled by leakage of gold solutions or hidden evaluation information, and
task quality issues, including misleading problem statements and improperly scoped tests. These issues can… See the full description on the dataset page: https://huggingface.co/datasets/opencompass/SWEBench-Pro-Verified.Nemotron-RL-Agentic-SWE-Pivot-v1
Dataset Description:
The SWE-RL dataset provides GitHub issues for training and validating real-world software engineering agents using the OpenHands environment in NeMo Gym. The dataset is a refactored version of the SWE-Gym and R2E-Gym datasets to support the NeMo Gym input format.
This dataset is released as part of NVIDIA NeMo Gym, a framework for building reinforcement learning environments to train large language models. NeMo Gym contains a growing collection of training… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-SWE-Pivot-v1.swebench-localisation
Finding the file: localisation on SWE-bench Verified
Given a GitHub issue, which file do you have to change? This is the retrieval step every coding
agent performs before it writes a patch, and none of the leaderboards score it separately. SWE-bench's five
leaderboards all score % Resolved, which folds localisation and patch-writing into one
number.
This bundle is that step measured on its own, on all 500 instances of SWE-bench Verified, with a
floor. The write-up is
Finding… See the full description on the dataset page: https://huggingface.co/datasets/RiverRider/swebench-localisation.swe-prbench
SWE-PRBench
Benchmarking AI Code Review Quality Against Human Pull Request Feedback
Blog: Read the blog
GitHub Repository: View the code
arXiv Paper: View the paper
Overview
SWE-PRBench is a benchmark of 350 pull requests with human-annotated
ground truth for evaluating whether LLMs can identify the same issues
that real human reviewers flag in production code.
Existing benchmarks like SWE-Bench measure whether models can produce
correct code. SWE-PRBench… See the full description on the dataset page: https://huggingface.co/datasets/foundry-ai/swe-prbench.qwen36-kquant-offload-mtp-swebench-lite100-results
Qwen3.6 K-Quant Offload MTP SWE-bench Lite 100 Results
This dataset contains the complete 5-model x 100-prompt runtime benchmark artifacts plus a detailed statistical analysis layer.
Primary conclusion: hot30/cold30 was the best decode-throughput run, while Q4_K_M had the best total wall clock. The ATX hot30/cold30 quantization significantly outperformed both Q4_K_M and Q3_K_XL on paired decode throughput, but Q4_K_M remains the elapsed-time control.
The ATX/K3 hot10, hot20, and… See the full description on the dataset page: https://huggingface.co/datasets/jakeatx/qwen36-kquant-offload-mtp-swebench-lite100-results.harbor-swesmith-rl-artifacts
Harbor SWE-Smith 强化学习数据产物
本数据集是 Harbor Qwen 工具调用代码智能体强化学习项目使用的冻结任务集，服务于 GRPO、原生价值模型/GAE PPO、训练过程诊断和统一协议评测。
项目已于 2026 年 8 月 30 日完成 P0 评测并进入阶段性归档。本数据集用于保留实验所依赖的数据切分、任务执行文件和审计信息，不代表新的通用代码能力基准。
数据概况
切分
任务数
训练集
187
验证集
42
测试集
38
合计
267
数据覆盖 89 个上游代码仓库。三个切分之间同时执行任务标识和仓库级隔离检查。
正式数据集名称：
swesmith-curated-grpo-267-v1
冻结切分的语义摘要：
ae5df9a3f4a3fc8af44fac420b36529e283839e1bd3de9daba65d5bcda51447d
该值来自 split-manifest.json 的 sha256 字段，用于标识切分语义，不等同于该文件本身的字节级… See the full description on the dataset page: https://huggingface.co/datasets/keryszhan/harbor-swesmith-rl-artifacts.SWE-Compass
🧠 SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models
Current evaluations of LLMs for software engineering are limited by a narrow range of task categories, a Python-centric bias, and insufficient alignment with real-world development workflows.To bridge these gaps, SWECompass establishes a high-coverage, multi-dimensional, and production-aligned evaluation framework:
✨ Covers 8 software… See the full description on the dataset page: https://huggingface.co/datasets/Kwaipilot/SWE-Compass.swe-agent-tool-rubrics-860
SWE Agent 逐 turn 工具调用评判数据集（860 个决策点）
本数据集来自 2026-08-06 的一次实验：**从真实 SWE agent 轨迹中归纳"怎么判断一次工具调用的好坏"**。
包含两个文件：
文件
行数
大小
内容
cases.jsonl
860
5.0 MB
决策点原始数据（题目、历史、两个候选命令、执行结果、现役判官打分）
map_io.jsonl
860
9.6 MB
每个决策点喂给 GPT-5.6 的完整 prompt 原文与完整回复
两个文件通过 case_id 一一对应。
背景：为什么是"按动作分类"而不是"按工具分类"
轨迹来自 slime 的 minimal harness，该 harness 只暴露一个工具 bash
（slime/agent/harness/minimal.py 里的 BASH_TOOL），全部 328,270 次调用的工具名都是 bash。
所以"不同工具用不同 rubric"无法按工具名实现，只能按命令在干什么分类。… See the full description on the dataset page: https://huggingface.co/datasets/MasterVito/swe-agent-tool-rubrics-860.opus-gpt-swe-frontier-core
SWE Base
Repository-level software engineering trajectories for training coding agents.
2,459 chat trajectories · 48,499 API calls · $837.57 recorded generation cost
SWE-bench · debugging · patching · tools · agents
Overview
SWE Base is a software-engineering dataset centered on real repository issues. Each training example gives an agent a problem statement and captures the multi-turn process of inspecting a codebase, reasoning about a bug… See the full description on the dataset page: https://huggingface.co/datasets/Roman1111111/opus-gpt-swe-frontier-core.swe_v0.1_jsonl_wo_mlang_large100_wo_v0.0_deltagLogics-SWE-Env-2.5K
Logics-SWE-Env-2.5K
2,553 software engineering task instances · 1,771 repositories · 4 programming languages
🤗 Related model: Logics-SWE-Qwen3.6-27B
📄 Paper: One to More, More to One
💻 GitHub: AgenticBigBang
Overview
What is this dataset?
Logics-SWE-Env-2.5K is a collection of repository-level software engineering tasks for research on coding agents and environment-based reinforcement learning. It contains 2,553 unique task instances from 1,771… See the full description on the dataset page: https://huggingface.co/datasets/Logics-MLLM/Logics-SWE-Env-2.5K.swe-bench-lite-agent-traces-v14
AgentBRANE SWE-bench Lite Agent Traces v14
This release contains the 1,890 harness-native agent traces selected by the sealed SWE-bench Lite v14 publication record (1,379/1,890 resolved, 73.0%). It includes Claude Code, Codex, and Pi sessions across seven models and three replicates. No internal research notes are included.
Load the observation table:
from datasets import load_dataset
traces = load_dataset("melissapan/swe-bench-lite-agent-traces-v14", split="train")
Each row… See the full description on the dataset page: https://huggingface.co/datasets/melissapan/swe-bench-lite-agent-traces-v14.swebench-cc-traces-rawswebench-cc-traces-anonswe_bench_qiskit_testSWE-FeaturesThe columns repo, pr_number, title, body, changed_files are self-explanatory. The other columns are:
changed_files_count: How many files are modified in this pull request (only js and py files are counted)
meets_criteria: It means whether gpt-4o-mini thinks this pull request meets the following criteria: (a) the title and body clearly describe the PR; (b) the PR introduces new functionality rather than fixing bugs; and (c) the PR is independent and not
obviously linked to other issues.… See the full description on the dataset page: https://huggingface.co/datasets/letta-ai/SWE-Features.kimicode_swetogether_traces
kimi-code × SWE-Together agentic traces
This is a dataset generated by a real agentic coding workload: SWE-Together tasks executed by the kimi-code agent, with every LLM call captured at the wire level. It is intended primarily as an inference-serving workload (in the spirit of Inferact/codex_swebenchpro_traces): full multi-turn sessions preserve the request structure — growing contexts, shared prefixes, tool calls — that drives KV-cache behavior in production serving.… See the full description on the dataset page: https://huggingface.co/datasets/verda-research/kimicode_swetogether_traces.swebench-verified-deepseek-v4-flash-failure-analysis
SWE-bench Verified runs & failure analysis — DeepSeek-V4-flash (local) × mini-swe-agent
Per-instance analysis of SWE-bench Verified runs of a locally-served DeepSeek-V4-flash model
driven by mini-swe-agent, graded with the official
SWE-bench harness. Each instance carries the full agent trajectory, a readable transcript, the
submitted patch, the harness test output, deterministic metrics, and a hand-verified qualitative
root-cause diagnosis.
Current numbers (resolve rates… See the full description on the dataset page: https://huggingface.co/datasets/daaain/swebench-verified-deepseek-v4-flash-failure-analysis.SWEUniverse-SizeMatched-45-ReliableTests-20260522
SWEUniverse Size-Matched 45 Repo Reliable Tests
This dataset contains the reliable-test universe rows collected for the 45 repositories in selected_repos_1000_size_matched.tsv.
Contents
data/reliable_tests.jsonl: one strict-stability reliable-test row per completed repo at the 600s timeout bucket. Rows include actual stable_passing, stable_failing, excluded_tests, raw_log_refs, parser metadata, command, commit, and replica counts.
data/snapshot_observations.jsonl:… See the full description on the dataset page: https://huggingface.co/datasets/VmaxRL/SWEUniverse-SizeMatched-45-ReliableTests-20260522.swebench-minimax-traces
swebench-minimax-traces
Complete HTTP-level agentic traces from running swebench benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 500
Multi-turn sessions (2+ LLM calls): 357
Total records: 22424
Total LLM requests: 11249
Format
Raw JSONL traces from the instrumented proxy.… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-minimax-traces.swe-smith-frozen-trajectories-openai
SWE-Smith Frozen Trajectories — OpenAI Wire Format
This dataset is the OpenAI chat-completions wire-format release of
reflectio/swe-smith-frozen-trajectories,
derived from the tool split of
SWE-bench/SWE-smith-trajectories.
It is a serving-performance workload for realistic multi-turn coding-agent
histories. It can be used to measure request throughput, input/output token
throughput, TTFT, TPOT, streaming behavior, and prefix-cache reuse. It is not
a coding-correctness… See the full description on the dataset page: https://huggingface.co/datasets/reflectio/swe-smith-frozen-trajectories-openai.swe-smith-frozen-trajectories
SWE-Smith Frozen Trajectories
This dataset is a serving-performance workload derived from the tool split of
SWE-bench/SWE-smith-trajectories.
It is designed for measuring throughput, request rate, time to first token,
inter-token latency, and prefix-cache behavior with realistic multi-turn coding
agent histories.
It is not a coding-correctness benchmark. The tested model's responses are
not executed or scored.
Processing
Keep trajectories generated by… See the full description on the dataset page: https://huggingface.co/datasets/reflectio/swe-smith-frozen-trajectories.swev-trm-trajectories-25models
SWE-Bench Verified TRM Trajectories (25 Models, Verified Labels)
Trajectories from 25 LLMs attempting SWE-Bench Verified tasks, formatted for
training a Trajectory Reward Model (TRM). Each record is one model's full
multi-turn attempt at one task, labeled with the real SWE-bench harness
verdict (scores.resolved).
Splits
Split
Records
Tasks
Pos
Neg
train
10,107
405
6,108
3,999
val
2,366
95
1,440
926
Train/val are task-disjoint (stable hash on task_id… See the full description on the dataset page: https://huggingface.co/datasets/tarsur385/swev-trm-trajectories-25models.swerebench-traces-raw-source-verification-enhanced-20260617
SWE-rebench Raw Source Verification Enhanced 20260617
This is a private raw source dataset for building refined mini-swe-agent SFT datasets. It is intentionally not tokenized and intentionally preserves source data plus metadata for downstream filtering, masking, weighting, and audit. Do not treat every row as a clean endpoint solve.
Download
The full dataset directory is uploaded as a single compressed archive:
hf download… See the full description on the dataset page: https://huggingface.co/datasets/eewer/swerebench-traces-raw-source-verification-enhanced-20260617.SWE-rebench-filtered
Dataset Summary
SWE-rebench-OpenHands-Trajectories is a dataset of multi-turn agent trajectories for software engineering tasks, collected
using Qwen/Qwen3-Coder-480B-A35B-Instruct with OpenHands (v0.54.0) agent scaffolding.
This dataset captures complete agent execution traces as they attempt to resolve real GitHub issues from
nebius/SWE-rebench.
Each trajectory contains the agent's step-by-step reasoning, actions, and environmental observations.
Metric… See the full description on the dataset page: https://huggingface.co/datasets/jessezhaoxizhang/SWE-rebench-filtered.swebench-sonnet-traces
swebench-sonnet-traces
Complete HTTP-level agentic traces from running swebench_sonnet benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 122
Multi-turn sessions (2+ LLM calls): 69
Total records: 2748
Total LLM requests: 1374
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-sonnet-traces.qwen3-orthdion-sweep