CoolFace
Datasetpublic

experiential-labs/wmo-bird-sql-traces

bird-sql — real agent-environment traces Text-to-SQL over real SQLite databases: the agent explores a copy of the task's database and schema, then submits a SQL query. Every trace is a REAL run: an LLM agent stepping against the actual benchmark environment, with each transition (tool call → true environment observation) recorded as OpenTelemetry GenAI spans (traces.otel.jsonl, one span per line). Captured by world-model-harness's environment-capture package, which also holds… See the full description on the dataset page: https://huggingface.co/datasets/experiential-labs/wmo-bird-sql-traces.

sourceHugging Facecc-by-sa-4.0updated 2mo agoView on Hugging Face
0likes522downloads
Dataset Card

bird-sql — real agent-environment traces

Text-to-SQL over real SQLite databases: the agent explores a copy of the task's database and schema, then submits a SQL query.

Every trace is a REAL run: an LLM agent stepping against the actual benchmark environment, with each transition (tool call → true environment observation) recorded as OpenTelemetry GenAI spans (traces.otel.jsonl, one span per line). Captured by world-model-harness's environment-capture package, which also holds the adapter, capture scripts, and per-corpus provenance: see `packages/environment-capture/bird-sql/`.

License and attribution

Derived from bird-bench mini-dev (CC BY-SA 4.0); this corpus is redistributed under the same terms (cc-by-sa-4.0). The trace text embeds task data and environment output from the upstream benchmark — keep this attribution if you redistribute.

Contents

  • —traces.otel.jsonl — the trace corpus (OTel GenAI spans, one JSON object per line)
  • —data/ — task index (train/test splits: prompts + task metadata)
  • —gold/ — per-task gold sidecars (graders read these; never staged into agent workspaces)
  • —schemas/ — database DDL per task database

Using it

python
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    "experiential-labs/wmh-bird-sql-traces", "traces.otel.jsonl", repo_type="dataset"
)

or, from a world-model-harness checkout:

bash
uv run wmh download bird-sql