datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Long-Horizon-Terminal-Bench
Long-Horizon Terminal-Bench (LHTB)
LHTB is a 46-task benchmark for measuring how well LLM agents sustain useful
work in a containerized terminal over hundreds of steps. Unlike short-horizon
coding benchmarks where an agent writes one artifact and stops, LHTB drops the agent
into a stateful environment and grades it with hidden, rebuild-from-artifact
verifiers — self-reported progress does not count.
📝 Blog: https://zli12321.github.io/LHTB/
🏆 Leaderboard:… See the full description on the dataset page: https://huggingface.co/datasets/IntelligenceLab/Long-Horizon-Terminal-Bench.Long-Horizon-Execution
Long Horizon Execution
This project contains the dataset accompanying the paper "The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs"
Abstract
Does continued scaling of large language models (LLMs) yield diminishing returns? Real-world value often stems from the length of task an agent can complete. We start this work by observing the simple but counterintuitive fact that marginal gains in single-step accuracy can compound into exponential… See the full description on the dataset page: https://huggingface.co/datasets/arvindh75/Long-Horizon-Execution.long-horizon-traj
long-horizon-traj
Long-horizon agent trajectories with multi-step planning and constraints
Dataset Description
This dataset contains agent trajectories for multi-turn tool use tasks, including reasoning traces, tool calls, and responses.
Dataset Structure
The dataset is organized by model name, with each model having separate JSONL files for different experimental passes.
long-horizon-traj/
├── model-1/
│ ├── pass@1.jsonl
│ ├── pass@2.jsonl
│ └──… See the full description on the dataset page: https://huggingface.co/datasets/ToolGym/long-horizon-traj.k2-horizon-tiny-cpu-repro-v1
K2-Horizon MoVA tiny random CPU fixture
Complete untrained K2HorizonForCausalLM, not Moonshot Kimi despite the K2 name.
Architecture source: IFM/K2-Horizon-MoVA-36B-A4B at 05cab0a4d7150c1c460a000b37ff40cc1af2feaa.
No pretrained weights, original tokenizer, training data, paid GPU or cloud compute used.
Complete text-only K2HorizonForCausalLM, not Kimi: three-layer dense prefix followed by two real MoVA+MoE layers, grouped RMSNorm, sigmoid top-k routing with selection-only bias… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/k2-horizon-tiny-cpu-repro-v1.long-horizon
ToolGym Long-Horizon Dataset
Dataset Description
This dataset contains long-horizon trajectories and evaluations for the ToolGym benchmark.
Dataset Structure
long-horizon/
├── traj/ # Agent trajectories (JSONL format)
│ ├── gpt-5.2/
│ │ ├── pass@1.jsonl
│ │ ├── pass@2.jsonl
│ │ └── pass@3.jsonl
│ ├── claude-opus-4.5/
│ └── ...
└── eval/ # Evaluation results (JSONL format)
├── claude-opus-4.5/
│ ├──… See the full description on the dataset page: https://huggingface.co/datasets/masculine/long-horizon.short-horizon-eval
short-horizon-eval
Evaluation results for short-horizon agent trajectories
Dataset Description
This dataset contains evaluation results for agent trajectories, including quality assessments and performance metrics.
Dataset Structure
The dataset is organized by model name, with each model having separate JSONL files for different experimental passes.
short-horizon-eval/
├── model-1/
│ ├── pass@1.jsonl
│ ├── pass@2.jsonl
│ └── pass@3.jsonl
├── model-2/
│… See the full description on the dataset page: https://huggingface.co/datasets/ToolGym/short-horizon-eval.short-horizon-traj
short-horizon-traj
Short-horizon agent trajectories showing reasoning traces and tool use
Dataset Description
This dataset contains agent trajectories for multi-turn tool use tasks, including reasoning traces, tool calls, and responses.
Dataset Structure
The dataset is organized by model name, with each model having separate JSONL files for different experimental passes.
short-horizon-traj/
├── model-1/
│ ├── pass@1.jsonl
│ ├── pass@2.jsonl
│ └──… See the full description on the dataset page: https://huggingface.co/datasets/ToolGym/short-horizon-traj.long-horizon-memorization
ComposeCL Datasets
This repository contains the three 100-task memorization datasets from the paper Continual Learning Mechanisms Compose for Long-Horizon Memorization (arXiv: 2609.06986).
Project page: https://compose-cl.github.io/
Code: https://github.com/cozheyuanzhangde/compose-cl
Datasets
Dataset
Path
Tasks
Items/task
Construction
Symbol-QA
data/synthetic_qa/symbol_qa
100
100
random six-character keys mapped to random four-character values… See the full description on the dataset page: https://huggingface.co/datasets/cozzyde/long-horizon-memorization.long-horizon-eval
long-horizon-eval
Evaluation results for long-horizon agent performance
Dataset Description
This dataset contains evaluation results for agent trajectories, including quality assessments and performance metrics.
Dataset Structure
The dataset is organized by model name, with each model having separate JSONL files for different experimental passes.
long-horizon-eval/
├── model-1/
│ ├── pass@1.jsonl
│ ├── pass@2.jsonl
│ └── pass@3.jsonl
├── model-2/
│ └──… See the full description on the dataset page: https://huggingface.co/datasets/ToolGym/long-horizon-eval.short-horizon
ToolGym Short-Horizon Dataset
Dataset Description
This dataset contains short-horizon trajectories and evaluations for the ToolGym benchmark.
Dataset Structure
short-horizon/
├── traj/ # Agent trajectories (JSONL format)
│ ├── claude-3.5/
│ │ ├── pass@1.jsonl
│ │ ├── pass@2.jsonl
│ │ └── pass@3.jsonl
│ ├── deepseek-v3.2/
│ └── ...
└── eval/ # Evaluation results (JSONL format)
├── claude-3.5/
│ ├──… See the full description on the dataset page: https://huggingface.co/datasets/masculine/short-horizon.Long-Horizon-Legal-Reasoning
Long-Horizon Legal Reasoning with Dispute Resolution
A benchmark for long-horizon legal reasoning over Ontario employment-law disputes. Each
sample presents the materials of a real-world-style dispute and asks a model to produce a
full legal analysis, which is scored against an expert gold answer with an LLM-as-judge.
The benchmark isolates two axes: how the case is presented (a chronological Timeline
vs. raw Documents) and which retrieval/tool approach the model uses (zero-shot… See the full description on the dataset page: https://huggingface.co/datasets/k-w-k/Long-Horizon-Legal-Reasoning.mid-horizon-drift-v0.1b
What this dataset tests
Mid-range trends can lie.
Mix shifts.
Definitions shift.
Leading indicators speak first.
Why it exists
Models over-trust blended mid-horizon metrics.
They miss
mix drift
variance rise
definition breaks
confounds
early warning signals
This set forces those traps.
Data format
Each row contains
mid_horizon_context
user_message
drift_pressure
constraints
failure_modes_to_avoid
target_behaviors
gold_checklist
Feed the… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/mid-horizon-drift-v0.1b.long-horizon-drift-v0.1c
What this dataset tests
Long arcs bend.
Past stability can hide future risk.
Why it exists
Long plans fail when drift accumulates.
This set checks whether you
detect slow shifts
reject frozen baselines
project forward risk
set milestones
Data format
Each row contains
long_horizon_context
user_message
drift_pressure
constraints
failure_modes_to_avoid
target_behaviors
gold_checklist
Feed the model
long_horizon_context
user_message… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/long-horizon-drift-v0.1c.ai-goal-failure-horizon-and-realignment-routing-v0.1What this dataset is
Predicts how soon goal drift becomes a hard failure
Names the realignment window before collapse
Forces an intervention choice with triggers and monitoring
Inputs
setting
env_shift_event
observed_drift_markers
goal_representation_summary
behavioral_deviation_summary
system_constraints
intervention_options
Gold fields in the CSV
failure_mode
estimated_failure_horizon_steps
realignment_window_steps
gold_intervention_choice
realignment_trigger_conditions… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/ai-goal-failure-horizon-and-realignment-routing-v0.1.
