CoolFace
Datasetpublic

experiential-labs/wmo-crmarena-traces

crmarena — real agent-environment traces Professional CRM analytics over a realistic Salesforce org snapshot: case routing, handle-time analytics, and entity disambiguation via SQL. Every trace is a REAL run: an LLM agent stepping against the actual benchmark environment, with each transition (tool call → true environment observation) recorded as OpenTelemetry GenAI spans (traces.otel.jsonl, one span per line). Captured by world-model-harness's environment-capture package, which… See the full description on the dataset page: https://huggingface.co/datasets/experiential-labs/wmo-crmarena-traces.

sourceHugging Facecc-by-nc-4.0updated 2mo agoView on Hugging Face
0likes94downloads
Dataset Card

crmarena — real agent-environment traces

Professional CRM analytics over a realistic Salesforce org snapshot: case routing, handle-time analytics, and entity disambiguation via SQL.

Every trace is a REAL run: an LLM agent stepping against the actual benchmark environment, with each transition (tool call → true environment observation) recorded as OpenTelemetry GenAI spans (traces.otel.jsonl, one span per line). Captured by world-model-harness's environment-capture package, which also holds the adapter, capture scripts, and per-corpus provenance: see `packages/environment-capture/crmarena/`.

License and attribution

Derived from Salesforce CRMArena (CC BY-NC 4.0); this corpus is redistributed under the same terms (cc-by-nc-4.0). The trace text embeds task data and environment output from the upstream benchmark — keep this attribution if you redistribute.

Contents

  • —traces.otel.jsonl — the trace corpus (OTel GenAI spans, one JSON object per line)
  • —data/ — task index (train/test splits: prompts + task metadata)
  • —gold/ — per-task gold sidecars (graders read these; never staged into agent workspaces)

Using it

python
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    "experiential-labs/wmh-crmarena-traces", "traces.otel.jsonl", repo_type="dataset"
)

or, from a world-model-harness checkout:

bash
uv run wmh download crmarena