datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
apex-agents
APEX–Agents
APEX–Agents is a benchmark from Mercor for evaluating whether AI agents can execute long-horizon, cross-application professional services tasks. Tasks were created by investment banking analysts, management consultants, and corporate lawyers, and require agents to navigate realistic work environments with files and tools (e.g., docs, spreadsheets, PDFs, email, chat, calendar).
Tasks: 480 total (160 per job category)
Worlds: 33 total (10 banking, 11 consulting, 12… See the full description on the dataset page: https://huggingface.co/datasets/mercor/apex-agents.TIR-Bench
TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoning
Introduction:
TIR-Bench is a comprehensive benchmark designed to evaluate the "thinking-with-images" capabilities of Multimodal Large Language Models (MLLMs), addressing a gap left by existing benchmarks like Visual Search which only test basic operations. As models like OpenAI o3 begin to intelligently create and operate tools to transform images for problem-solving, TIR-Bench provides 13… See the full description on the dataset page: https://huggingface.co/datasets/Agents-X/TIR-Bench.apex-agents
APEX–Agents
APEX–Agents is a benchmark from Mercor for evaluating whether AI agents can execute long-horizon, cross-application professional services tasks. Tasks were created by investment banking analysts, management consultants, and corporate lawyers, and require agents to navigate realistic work environments with files and tools (e.g., docs, spreadsheets, PDFs, email, chat, calendar).
Tasks: 480 total (160 per job category)
Worlds: 33 total (10 banking, 11 consulting, 12… See the full description on the dataset page: https://huggingface.co/datasets/idleengine/apex-agents.SWITCH
SWITCH: Benchmarking Modeling and Handling of Tangible Interfaces in Long-horizon Embodied Scenarios
[arXiv]
[leaderboard]
[dataset]
[PDF]
Dataset Summary
SWITCH (Semantic World Interface Tasks for Control & Handling) is a multimodal embodied-interaction benchmark for understanding, modeling, and evaluating actions over Tangible Control Interfaces (TCIs) in egocentric real-world scenarios.
TCIs include everyday interfaces such as appliance panels, lighting… See the full description on the dataset page: https://huggingface.co/datasets/BAAI-Agents/SWITCH.apex-agents
APEX–Agents
APEX–Agents is a benchmark from Mercor for evaluating whether AI agents can execute long-horizon, cross-application professional services tasks. Tasks were created by investment banking analysts, management consultants, and corporate lawyers, and require agents to navigate realistic work environments with files and tools (e.g., docs, spreadsheets, PDFs, email, chat, calendar).
Tasks: 480 total (160 per job category)
Worlds: 33 total (10 banking, 11 consulting, 12 law)… See the full description on the dataset page: https://huggingface.co/datasets/neyralabs/apex-agents.apex-agents
APEX–Agents
APEX–Agents is a benchmark from Mercor for evaluating whether AI agents can execute long-horizon, cross-application professional services tasks. Tasks were created by investment banking analysts, management consultants, and corporate lawyers, and require agents to navigate realistic work environments with files and tools (e.g., docs, spreadsheets, PDFs, email, chat, calendar).
Tasks: 480 total (160 per job category)
Worlds: 33 total (10 banking, 11 consulting, 12 law)… See the full description on the dataset page: https://huggingface.co/datasets/lobinni/apex-agents.apex-agents
APEX–Agents
APEX–Agents is a benchmark from Mercor for evaluating whether AI agents can execute long-horizon, cross-application professional services tasks. Tasks were created by investment banking analysts, management consultants, and corporate lawyers, and require agents to navigate realistic work environments with files and tools (e.g., docs, spreadsheets, PDFs, email, chat, calendar).
Tasks: 480 total (160 per job category)
Worlds: 33 total (10 banking, 11 consulting, 12 law)… See the full description on the dataset page: https://huggingface.co/datasets/abridges/apex-agents.