zzh237/agentbake-traces

AgentBake Trace Library

Execution-trace corpus for the AgentBake benchmark (A Personalization Layer and Benchmark for Heterogeneous Agents). 100 heterogeneous agents spanning seven frameworks (AutoGen, CrewAI, LangChain, LangGraph, LlamaIndex, PydanticAI, Strands), each contributing ~10 recorded multi-turn scenarios.

Contents

good_by_framework/
  autogen/<agent>/scenario_XXX/
    *__trace_sequence.json   # per-step activations: input text, output, tools, timing
    *__topology.json         # agent graph: nodes, roles, declared capabilities
    *__prebuilt_eval.json    # LLM-judge scores (coherence, goal success, ...)
    selection_metadata.json
  crewai/ ... langchain/ ... langgraph/ ... llamaindex/ ... pydanticai/ ... strands/

~4,900 files, ~6.2 GB. 16/14/15/30/5/10/10 agents per framework.

Provenance

Traces were generated by executing rebuilt open-source agents (149-agent adapter library, pinned upstream repos + commit SHAs in the companion code repo) on synthetic task scenarios, with Qwen3-32B (qwen.qwen3-32b-v1:0, via Amazon Bedrock) as the agent backend LLM and as the prebuilt evaluator judge. The trace content (inputs, outputs, topologies, judge scores) is generated data; upstream agent code is not included here — see the code repository for the adapter library and per-repo licenses.

Usage

from huggingface_hub import snapshot_download
snapshot_download("zzh237/agentbake-traces", repo_type="dataset",
                  local_dir="data/multiagent_traces")

Then run the AgentBake benchmark from the code repo: https://github.com/zzh237/AgentBake

Intended use

Research on agent personalization, orchestration-policy learning, and benchmark evaluation. Scenarios are synthetic; no real user data.