datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
epoch_ai_swebench_verified
Epoch AI SWE-bench Verified Traces
Complete public trace archives and an analysis-ready Parquet conversion of Epoch AI's SWE-bench Verified evaluations.
Contents
34 published evaluation runs covering 16,456 traces (484 SWE-bench instances per run).
data/: loadable Parquet data, one exact trace per row.
original/: the byte-identical .eval archives published by Epoch AI.
run_metadata/: non-sample files from each .eval archive (header.json, summaries, reductions… See the full description on the dataset page: https://huggingface.co/datasets/naderalfares/epoch_ai_swebench_verified.SWE-Bench-ProMax
🚀 SWE-Bench-ProMax
A Contamination-Resistant, Multi-Language Benchmark for Repository-Level Issue Resolution
📄 Paper ·
📊 Dataset ·
⚙️ Evaluation ·
📬 Contact
SWE-Bench-ProMax is a multi-language benchmark for repository-level software issue resolution, accepted to COLM 2026. All task instances are collected from real-world GitHub issues created after 2025, substantially reducing the risk of training-data contamination for contemporary language models. Each instance provides a task… See the full description on the dataset page: https://huggingface.co/datasets/swe-bench-promax/SWE-Bench-ProMax.swe-bench-lite-agent-traces-v14
AgentBRANE SWE-bench Lite Agent Traces v14
This release contains the 1,890 harness-native agent traces selected by the sealed SWE-bench Lite v14 publication record (1,379/1,890 resolved, 73.0%). It includes Claude Code, Codex, and Pi sessions across seven models and three replicates. No internal research notes are included.
Load the observation table:
from datasets import load_dataset
traces = load_dataset("melissapan/swe-bench-lite-agent-traces-v14", split="train")
Each row… See the full description on the dataset page: https://huggingface.co/datasets/melissapan/swe-bench-lite-agent-traces-v14.swebench-verified-deepseek-v4-flash-failure-analysis
SWE-bench Verified runs & failure analysis — DeepSeek-V4-flash (local) × mini-swe-agent
Per-instance analysis of SWE-bench Verified runs of a locally-served DeepSeek-V4-flash model
driven by mini-swe-agent, graded with the official
SWE-bench harness. Each instance carries the full agent trajectory, a readable transcript, the
submitted patch, the harness test output, deterministic metrics, and a hand-verified qualitative
root-cause diagnosis.
Current numbers (resolve rates… See the full description on the dataset page: https://huggingface.co/datasets/daaain/swebench-verified-deepseek-v4-flash-failure-analysis.agent-trajectories-swe-bench-test-minus-verified
Agent Trajectories: SWE-bench Test \ Verified — Mixed Teachers (gpt-5.2 / gpt-5-mini)
Summary
Full multi-turn agent trajectories collected from the SWE-bench Test minus Verified split
(i.e., SWE-bench Test instances that are not part of SWE-bench Verified).
Intended for SFT of agent models on coding tasks.
Data Collection
Each trajectory was produced by a GT-aware lookahead agent that, at every turn:
Sampled a candidate response from both gpt-5.2 and… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/agent-trajectories-swe-bench-test-minus-verified.swe-bench-trajectory-quality-subsets
SWE-bench Trajectory Quality Subsets
Curated subsets of nebius/SWE-rebench-openhands-trajectories constructed using the v3 quality scoring framework for fine-tuning evaluation.
Subsets Overview
Subset
Size
Selection
Mean Score
Resolved Rate
Purpose
Ablation-NoB2-500
500
Top 500 with Efficiency = B3 alone (drop B2 error_retry)
0.6410
100%
Ablation study
Ablation-NoB3-500
500
Top 500 with Efficiency = B2 alone (drop B3 step_count_ratio)
0.7253
100%
Ablation… See the full description on the dataset page: https://huggingface.co/datasets/davongluck/swe-bench-trajectory-quality-subsets.SWEbench-Verified-eval150-M2.7-Qwen3.5-9B-orch-7arms-2repeats-w32-20260920
SWE-bench Verified eval150 — M2.7 × Qwen3.5-9B, seven arms, two repeats, 32 concurrency
Campaign 2026-09-20. 14/14 independent full150 runs audited. Complete accuracy evidence.
Evaluation mode is orch: MiniMax-M2.7 orchestrator and the specified Qwen3.5-9B worker. Training mode is labeled independently. All runs use 32 concurrent episodes, 10GiB Docker sandboxes, four TP1 workers and one TP4/EP4 coordinator. Frozen regression-gated prompts, decoding and canonical verifier match… See the full description on the dataset page: https://huggingface.co/datasets/CharlieLLL/SWEbench-Verified-eval150-M2.7-Qwen3.5-9B-orch-7arms-2repeats-w32-20260920.swebench-minimax-traces
swebench-minimax-traces
Complete HTTP-level agentic traces from running swebench benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 500
Multi-turn sessions (2+ LLM calls): 357
Total records: 22424
Total LLM requests: 11249
Format
Raw JSONL traces from the instrumented proxy.… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-minimax-traces.swebench-sonnet-traces
swebench-sonnet-traces
Complete HTTP-level agentic traces from running swebench_sonnet benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 122
Multi-turn sessions (2+ LLM calls): 69
Total records: 2748
Total LLM requests: 1374
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-sonnet-traces.devstral-swe-bench-verified
Devstral SWE-bench Verified Trajectories
This dataset contains 4,000 mini-SWE-agent trajectories generated by
mistralai/Devstral-Small-2-24B-Instruct-2512 on a fixed 250-problem training split of
SWE-bench Verified. There are 16 independently seeded rollouts per problem.
Every row preserves the exact request prefix mini-SWE-agent supplied before Devstral's
first generation, the complete ordered rollout, a sanitized final patch, and termination
metadata. Official SWE-bench… See the full description on the dataset page: https://huggingface.co/datasets/suryadv/devstral-swe-bench-verified.swebench-deepseek-traces
swebench-deepseek-traces
Complete HTTP-level agentic traces from running swebench_deepseek benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 50
Multi-turn sessions (2+ LLM calls): 18
Total records: 542
Total LLM requests: 271
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-deepseek-traces.swebench-deepseek-traces-3
swebench-deepseek-traces-3
Complete HTTP-level agentic traces from running swebench_deepseek benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 46
Multi-turn sessions (2+ LLM calls): 26
Total records: 2154
Total LLM requests: 1077
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-deepseek-traces-3.swebench-deepseek-traces-2
swebench-deepseek-traces-2
Complete HTTP-level agentic traces from running swebench_deepseek benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 37
Multi-turn sessions (2+ LLM calls): 33
Total records: 2040
Total LLM requests: 1020
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-deepseek-traces-2.swebench-verified-kimi-k2p6-traces
SWE-bench Verified Kimi K2.6 Reasoning Traces
This dataset contains reasoning traces generated on princeton-nlp/SWE-bench_Verified using fireworks_ai/kimi-k2p6-high with a mini-swe-agent based harness. It is intended for research and distillation of software-engineering agents.
The repository is published with three configs because each table has a different schema:
raw_trajectories: one row per SWE-bench instance with the patch, sanitized result JSON, full trajectory JSON, message… See the full description on the dataset page: https://huggingface.co/datasets/MemoryAsModality/swebench-verified-kimi-k2p6-traces.swebench-minimax-traces-2
swebench-minimax-traces-2
Complete HTTP-level agentic traces from running swebench benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 250
Multi-turn sessions (2+ LLM calls): 210
Total records: 15198
Total LLM requests: 7623
Format
Raw JSONL traces from the instrumented proxy.… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-minimax-traces-2.swebench-sonnet-traces-2
swebench-sonnet-traces-2
Complete HTTP-level agentic traces from running swebench_sonnet benchmark tasks through an instrumented reverse proxy.
Each trace captures full request/response pairs including system prompts, user messages, assistant responses, tool calls and results, and token usage metadata.
Stats
Total sessions: 108
Multi-turn sessions (2+ LLM calls): 45
Total records: 2552
Total LLM requests: 1276
Format
Raw JSONL traces from the instrumented… See the full description on the dataset page: https://huggingface.co/datasets/sammshen/swebench-sonnet-traces-2.bullet-swebench-verified
Bullet on SWE-bench Verified — 479/500 = 95.8%
Results for the Bullet coding agent on all 500 instances of
SWE-bench Verified, graded by the official swebench.harness.run_evaluation
scorer. Every instance was attempted and graded; there are no empty patches.
479 / 500 resolved = 95.8%
Run with the Bullet harness on gpt-5.6-sol at high reasoning effort, one attempt per instance.
By repository
repository
resolved
django
223/231
96.5%
sympy
73/75
97.3%… See the full description on the dataset page: https://huggingface.co/datasets/daviddata1/bullet-swebench-verified.swe_doc_gen_locate_swebench_1500
SWE-Doc-Gen-Locate Dataset (SWE-Bench 1500 entries)
A dataset for evaluating an agent's ability to locate a target Python function/class based on its functionality description and add a docstring.
Task Description
Given: A Python repository and a description of a function/class (NO name, NO file path)
Agent must:
Search the codebase to find where the target function/class is defined
Read the implementation to understand its behavior
Generate and add an appropriate… See the full description on the dataset page: https://huggingface.co/datasets/synthetic-code-training/swe_doc_gen_locate_swebench_1500.swe-bench-arkts
SWE-bench ArkTS
SWE-bench-style tasks mined from ArkTS pull requests and direct commits in HarmonyOS/OpenHarmony projects. The dataset has two evaluator-specific splits and 21 tasks in total.
Split
Tasks
Oracle
failed2pass
14
Hvigor case-level unit tests: 50 FAIL_TO_PASS, 8,653 PASS_TO_PASS, no regressions
compile2pass
7
ArkTS compiler/API transitions: 5 diagnostic-removal tasks and 2 SDK/toolchain migrations
Loading
from datasets import… See the full description on the dataset page: https://huggingface.co/datasets/hreyulog/swe-bench-arkts.