Salesforce/tracelab-comprehend

tracelab COMPREHEND synthetic corpus

Twelve seeded synthetic long-horizon agent sessions (JSONL event streams, ~11 MB total) released with the paper "Parsing the Stream: A Live Trace Model for Long-Horizon Agents and Their Observers" (Pakhomov & Nijkamp, Salesforce AI Research; arXiv:2609.01466, https://arxiv.org/abs/2609.01466).

• —📄 Paper: https://arxiv.org/abs/2609.01466
• —💻 Code, generator, benchmarks, and traces (BSD-3-Clause): https://github.com/SalesforceAIResearch/tracelab
• —🤗 Dataset: this repository (CC-BY-4.0)

Each file is one session in the Claude Code JSONL trace format: tool calls and results, errors, file touches, and session metadata, generated to be distributionally realistic (tool-heavy, error-carrying) while containing no real data. The corpus backs the paper's auditable COMPREHEND replication: monitoring questions are generated mechanically from ledger ground truth, and the corpus regenerates byte-identically from seeds 201–212 with the released generator (https://github.com/SalesforceAIResearch/tracelab):

python -m tracelab.bench.synth_corpus --seeds 201-212

Code, benchmark harnesses, and all workbench traces: see the tracelab repository (https://github.com/SalesforceAIResearch/tracelab). Ground truth for turns, top tool, files, and latest ask is computed by the generator independently of the parsing pipeline (zero mismatches across all twelve sessions).

Data license: CC-BY-4.0 (the generator code is BSD-3-Clause; the corpus regenerates byte-identically from it).

Citation

@article{pakhomov2026parsing,
  title={Parsing the Stream: A Live Trace Model for Long-Horizon Agents and Their Observers},
  author={Pakhomov, Egor and Nijkamp, Erik},
  journal={arXiv preprint arXiv:2609.01466},
  year={2026}
}