datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
terminal-bench-2
Terminal-Bench-2.0 Beta
Welcome to Terminal-Bench-2.0! If you’re reading this you’re a member of the Terminal-Bench community that we’ve selected to get a sneak peek at the latest version of the benchmark.
Getting Started
First, clone Harbor (formerly “Sandboxes”):
git clone https://github.com/laude-institute/harbor.git
From inside the Harbor directory run:
uv sync
This will install Harbor, our new package for running agent evals.
You should now be able to run TB 2.0!… See the full description on the dataset page: https://huggingface.co/datasets/penfever/terminal-bench-2.Long-Horizon-Terminal-Bench
Long-Horizon Terminal-Bench (LHTB)
LHTB is a 46-task benchmark for measuring how well LLM agents sustain useful
work in a containerized terminal over hundreds of steps. Unlike short-horizon
coding benchmarks where an agent writes one artifact and stops, LHTB drops the agent
into a stateful environment and grades it with hidden, rebuild-from-artifact
verifiers — self-reported progress does not count.
📝 Blog: https://zli12321.github.io/LHTB/
🏆 Leaderboard:… See the full description on the dataset page: https://huggingface.co/datasets/IntelligenceLab/Long-Horizon-Terminal-Bench.terminal-bench
Terminal-Bench Dataset
This dataset contains tasks from Terminal-Bench, a benchmark for evaluating AI agents in real terminal environments. Each task is packaged as a complete, self-contained archive that preserves the exact directory structure, binary files, Docker configurations, and test scripts needed for faithful reproduction.
The archive column contains a gzipped tarball of the entire task directory.
Dataset Overview
Terminal-Bench evaluates AI agents on… See the full description on the dataset page: https://huggingface.co/datasets/ia03/terminal-bench.terminal-bench-pro
Terminal-Bench Pro
Overview
Terminal-Bench Pro is a systematic extension of the original Terminal-Bench, designed to address key limitations in existing terminal-agent benchmarks.
400 tasks (200 public + 200 private) across 8 domains: data processing, games, debugging, system admin, scientific computing, software engineering, ML, and security
Expert-designed tasks derived from real-world scenarios and GitHub issues
High test coverage with ~28.3 test cases per… See the full description on the dataset page: https://huggingface.co/datasets/alibabagroup/terminal-bench-pro.terminal-bench-2.1-qwen3.8-27b-traces
Terminal-Bench 2.1 traces: Qwen3.8-27B-GPTQ-4bit, xhigh / medium / low / off
Complete agent trajectories, verifier output, timing and token usage for all 89
Terminal-Bench 2.1 tasks run locally with
btbtyler09/Qwen3.8-27B-GPTQ-4bit
on 2× RTX 3090, plus the adaptive fallback reruns at lower reasoning effort.
Headline result: 62/89 (69.66%) at xhigh in a single clean pass.
Cumulative best-of across xhigh → medium → low → off fallbacks: 70/89 (78.65%).
The second number is not a… See the full description on the dataset page: https://huggingface.co/datasets/Lottolabs/terminal-bench-2.1-qwen3.8-27b-traces.Terminal-Bench-Hard
Terminal-Bench Hard
Terminal-Bench Hard is a set of 100 challenging terminal-based agent tasks.
The tasks cover software engineering, debugging, data processing, system
administration, security, scientific computing, and related command-line
workflows.
Contents
tasks/: runnable tasks in Harbor format.
metadata/tasks.parquet: searchable task metadata and instructions.
Each task directory contains task.toml, instruction.md, an
environment/ directory, and verifier… See the full description on the dataset page: https://huggingface.co/datasets/Zhongzhi1228/Terminal-Bench-Hard.terminal-bench-traces-localterminal-bench-2-verified
Terminal-Bench 2.0 Verified: Instruction & Environment Fix Version
中文版本
We conducted a comprehensive review of the entire Terminal-Bench 2.0 dataset and identified various issues. Both GLM-5 and Step 3.5-Flash were evaluated using this verified version.
This modified version addresses environment and instruction issues we discovered in Terminal-Bench 2.0. It includes two types of fixes:
Environment Fixes: Updated Dockerfiles and instructions to support Claude Code Agent runtime… See the full description on the dataset page: https://huggingface.co/datasets/harithoppil/terminal-bench-2-verified.terminalbench-trajectories
Terminal-Bench 2.0 Trajectories
Full agent trajectories from Terminal-Bench 2.0, a benchmark that evaluates AI coding agents on real-world terminal tasks. Each row is one trial: an agent attempting a task, with the complete step-by-step trace of messages, tool calls, and observations.
Explorer: yoonholee.com/web-apps/terminal-bench
Quick start
from datasets import load_dataset
import json
ds = load_dataset("yoonholee/terminalbench-trajectories", split="train")
#… See the full description on the dataset page: https://huggingface.co/datasets/yoonholee/terminalbench-trajectories.GPT-5-terminal-bench-2terminal-bench-3.0-ox-alpha
Ox Alpha on Terminal-Bench 3.0: pass@1 traces
Complete, sanitized agent trajectories from a one-attempt-per-task evaluation of the OpenRouter stealth model Ox Alpha (stealth/ox-alpha) on Terminal-Bench 3.0. The dataset is packaged for result review and agent-failure analysis.
Links
Terminal-Bench 3.0 dataset on Harbor Hub
Terminal-Bench repository
Terminal-Bench 3.0 announcement and methodology
Ox Alpha on OpenRouter (historical model page used for this… See the full description on the dataset page: https://huggingface.co/datasets/thomasip/terminal-bench-3.0-ox-alpha.terminal-bench-2-leaderboard
Terminal-Bench 2.0 Leaderboard Submissions
This repository accepts leaderboard submissions for Terminal-Bench 2.0.
How to Submit
Fork this repository
Create a new branch for your submission
Add your submission (a job or folder of jobs) under submissions/terminal-bench/2.0/<agent>__<model(s)>/
Open a Pull Request
Submission Structure
submissions/
terminal-bench/
2.0/
<agent>__<model>/
metadata.yaml # Required: agent and model info… See the full description on the dataset page: https://huggingface.co/datasets/CO-IR/terminal-bench-2-leaderboard.terminal-bench-mini
terminal-bench-mini
Fourteen of Terminal-Bench 2.0's ninety tasks, picked so that ranking agents on
the subset reproduces ranking them on the whole benchmark.
Running ninety tasks five times each is how the official leaderboard is built.
That is out of reach if you are comparing quant variants, fine-tunes or local
models on your own hardware. This subset turns a multi-day sweep into a few
hours.
Same approach as deepswe-mini:
take the published per-task results, rank the field… See the full description on the dataset page: https://huggingface.co/datasets/LocalLLaMA/terminal-bench-mini.terminal-bench-2-leaderboard-1
Terminal-Bench 2.0 Leaderboard Submissions
This repository accepts leaderboard submissions for Terminal-Bench 2.0.
How to Submit
Fork this repository
Create a new branch for your submission
Add your submission (a job or folder of jobs) under submissions/terminal-bench/2.0/<agent>__<model(s)>/
Open a Pull Request
Submission Structure
submissions/
terminal-bench/
2.0/
<agent>__<model>/
metadata.yaml # Required: agent and model info… See the full description on the dataset page: https://huggingface.co/datasets/antinomyhq/terminal-bench-2-leaderboard-1.terminal_bench_2_a1_issue_tasks_20260805_125430claude-sonnet-4-5-terminal-bench-2terminal_bench_2_tasktrove_dq_stack_pytest_step25_30b_a3b_20260730_053956
TaskTrove stack-pytest — training rollout traces (Qwen3-Coder-30B-A3B, step 25)
Terminus-2/Harbor rollouts recorded while training
laion/tasktrove-dq-stack-pytest-step25-30b-a3b
with SkyRL on the TaskTrove stack-pytest source.
One row per trial, holding that trial's last episode as an OpenAI-style conversations list, the
task instruction, the reward the verifier assigned (result), and the verifier's own stdout
(verifier_output).
Source run… See the full description on the dataset page: https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_stack_pytest_step25_30b_a3b_20260730_053956.terminal_bench_2_tasktrove_dq_unix_step10_30b_a3b_20260730_014756
terminal_bench_2_tasktrove_dq_unix_step10_30b_a3b
OpenCode agent trajectories from the TaskTrove DQ unix arm of a Qwen3-Coder-30B-A3B
agentic RL sweep, exported from the complete Harbor rollout artifact set.
Coverage
Built from the full trace_jobs prefix of run rl-tasktrove-dq-sweep-30b-qwen3-coder-30-20260727-082204-e42f1d
(12034 trial directories, 11937 of them scored).
quantity
value
scored trials (result.json)
11937
rows published
11937
coverage… See the full description on the dataset page: https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_unix_step10_30b_a3b_20260730_014756.terminal_bench_2_a1_issue_tasks_20260627_124529terminal-bench-2-leaderboard
Terminal-Bench 2.0 Leaderboard Submissions
This repository accepts leaderboard submissions for Terminal-Bench 2.0.
How to Submit
Fork this repository
Create a new branch for your submission
Add your submission (a job or folder of jobs) under submissions/terminal-bench/2.0/<agent>__<model(s)>/
Open a Pull Request
Submission Structure
submissions/
terminal-bench/
2.0/
<agent>__<model>/
metadata.yaml # Required: agent and model info… See the full description on the dataset page: https://huggingface.co/datasets/3rdn4/terminal-bench-2-leaderboard.terminal-bench-2terminal_bench_2_a1_issue_tasks_20260325_000804terminal-bench-pro-eval-trajectoriesterminal_bench_2_a1_issue_tasks_20260711_172919terminal_bench_2_tasktrove_dq_taco_step15_30b_a3b_20260729_222705
Agent trace dataset
OpenCode/Harbor rollout traces from the MarinSkyRL run
rl-tasktrove-dq-sweep-30b-qwen3-coder-30-20260726-235656-574ba8, exported with
make_and_upload_trace_dataset --episodes last (the last episode of each trial — the rollouts
the policy was trained on).
Coverage
Built from the complete trial set on durable object storage, not from a local evidence bundle.
quantity
value
trial directories on object storage
21711
trials with a… See the full description on the dataset page: https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_taco_step15_30b_a3b_20260729_222705.terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827
TaskTrove DQ unitsyn-python training traces (step 20, 30B-A3B)
Terminus-2 agent rollouts recorded while training
laion/tasktrove-dq-unitsyn-python-step20-30b-a3b
with SkyRL from Qwen/Qwen3-Coder-30B-A3B-Instruct.
Each row is the last episode of one trial: the full agent transcript, the task instruction, the
scalar reward, and the verifier's output.
Source run: rl-tasktrove-dq-sweep-30b-terminus2-qwen-20260725-163115-1ae770.
Coverage
This dataset is the complete… See the full description on the dataset page: https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827.GPT-5-nano-terminal-bench-2terminal_bench_2_tasktrove_dq_tulu3_personas_math_step13_30b_a3b_20260730_054033
terminal_bench_2_tasktrove_dq_tulu3_personas_math_step13_30b_a3b
OpenCode agent traces from the Iris RL run
rl-tasktrove-dq-sweep-30b-qwen3-coder-30-20260727-143750-b2bcd7, exported from the
run's Harbor trace_jobs artifacts (last episode per trial).
Coverage is complete for this run: all 15,740 trial directories were enumerated and every
trial that produced a result.json is present. The 89 trials without a result.json
never completed a scoreable episode and contribute no rows.… See the full description on the dataset page: https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_tulu3_personas_math_step13_30b_a3b_20260730_054033.terminal-bench-2-swe-tasksnemotron-terminal-8b-eval-terminal-bench-lite-concurrency-100