gavinlaw/agentic-vbench-calibration-trajectories

AgenticVBench volleyball calibration trajectories

Native raw agent traces from the calibration of two AgenticVBench understanding tasks, published so a reviewer can audit turn counts, prompt parity and the no-lookup rule independently rather than taking a summary on trust.

• —usc-wsu-2023-volleyball-block-timeline — 23 block points, two attributions each
• —byu-wsu-2023-volleyball-block-timeline — 18 block points, three attributions each

The tasks themselves, the answer keys, the scorers and the calibration write-ups live in the benchmark repository: https://github.com/PhiloLabs/agentic-vbench

What is here

Every trace is the CLI's own output, gzipped and otherwise unmodified.

<task>/codex-fresh/rollout.jsonl.gz          Codex CLI event stream (--json)
<task>/codex-fresh/stderr.txt.gz             the same run's stderr
<task>/opus-fresh/rollout*.stream-json.gz    Claude Code stream-json (one file per leg)
<task>/ablations/<name>.stream-json.gz       forced-answer ablation runs
usc-.../fable-interrupted/rollout...gz       a run that stopped when its model's
                                             credit pool ran out; archived unscored

MANIFEST.json gives, for every archive, the SHA256 of the .gz as served here and the SHA256 of the uncompressed stream inside it, so either can be verified:

sha256sum <file>.gz                  # matches sha256_gz
gunzip -c <file>.gz | sha256sum      # matches sha256_uncompressed

Runs and scores

Each agent ran in a workspace holding only the video and the instruction, with no prior artifact of the task reachable. The exact prompt is committed in the benchmark repository beside each task's rollouts, and referenced there by SHA256.

The BYU Opus run is two legs of one session: the first was cut off by a network drop and resumed in place, which is why rollout.leg1 and rollout.leg2 together make one 386-turn run.

What the traces contain

Assistant messages, reasoning summaries where the CLI emits them, every tool call and its result, and the frames the agents extracted and read (base64-inlined by the CLI — this is why the files compress poorly). Workspace paths from the machine the calibration ran on are visible; nothing else about that machine is.

Licence and provenance

The traces are released CC-BY-4.0. They describe agent behaviour on two publicly broadcast NCAA women's volleyball matches; the video assets themselves are not here (see gavinlaw/agentic-vbench-media), and the answer keys derive from the official NCAA rally-by-rally logs.