paulbatum/pareto-rail-rollouts

Pareto Rail benchmark rollouts

Full agent transcripts for every published run of the Pareto Rail level-generation benchmark: models one-shot a browser rail-shooter level from a written theme, and visitors rank the results blind on the public site. The benchmark's methodology, per-run provenance manifests (rendered prompt, sealed commits, gate results, measured cost), and the level source produced by each of these transcripts live in the main repository: github.com/paulbatum/pareto-rail.

Layout

runs/<run-id>/assignment.md                               # the rendered prompt the agent received
runs/<run-id>/stages/<stage>/<harness>/rollout.jsonl      # the harness-native session, opens in the trace viewer
runs/<run-id>/stages/<stage>/<harness>/events.jsonl.gz    # the harness's emitted event stream
runs/<run-id>/stages/<stage>/<harness>/final-message.md   # the agent's closing message

Each rollout.jsonl is a raw Claude Code, Codex, or Pi session file, so it opens directly in the Hub's agent trace viewer; assignment.md and final-message.md are plain markdown. Transcripts are exactly as captured: agent screenshots taken during the run are embedded as base64. rollouts.json here (also committed in the main repository under benchmark/manifests/) maps each run to its level, theme, and configuration and records the size and sha256 of every transcript's uncompressed bytes, so a download can be verified (after gunzip, for the gzipped event streams).

Runs

Rehearsal runs (never ranked; kept as failure examples)

Rehearsal runs are unattended generation attempts that were never part of the ranked pool. They are published exactly as captured, including the gate failures that ended them.

Diagnostics (not benchmark runs)

Diagnostics are preserved agent sessions that were stopped or otherwise invalid before producing a benchmark result. They are not ranked evidence.