akenginorhun/neurips-2026-evals

NeurIPS 2026 Agent Evaluation Dataset

This dataset contains evaluation results for various AI agents across multiple benchmarks.

Dataset Structure

The dataset is organized by model (as configs) with each benchmark as a split.

Each model/benchmark folder contains:

• —Main results file (.jsonl or .parquet format)
• —Summary statistics (.summary.json) - for models with metadata
• —Configuration file (.toml) - for models with metadata
• —Traces folder with execution traces (traces/traces.jsonl) - for models with metadata

Models

• —claude-opus: 9 benchmarks
• —gemini-31-pro: 9 benchmarks
• —gemma4-e4b: 7 benchmarks
• —gpt-54: 9 benchmarks
• —kimi-k25: 4 benchmarks
• —lfm-12b: 7 benchmarks
• —minimax-m25: 7 benchmarks
• —nemotron-nano-30b: 7 benchmarks
• —nemotron-nano-4b-fp8: 7 benchmarks
• —qwen-27b: 6 benchmarks
• —qwen-2b: 7 benchmarks
• —qwen-397b: 7 benchmarks
• —qwen-4b: 6 benchmarks
• —qwen-9b: 7 benchmarks
• —trinity-large: 7 benchmarks

Benchmarks

• —deepresearch
• —gaia
• —livecodebench
• —liveresearch
• —liveresearchbench
• —pinchbench
• —taubench
• —taubench_telecom
• —terminalbench
• —toolcall15

Usage

from datasets import load_dataset

# Load a specific model's results
dataset = load_dataset('akenginorhun/neurips-2026-evals', name='claude-opus')

# Load a specific benchmark for a model
dataset = load_dataset('akenginorhun/neurips-2026-evals', name='qwen-27b', split='gaia')