agentic-ptb/INDEX

AgentPTB checkpoint index

Every checkpoint produced by the AgentPTB driver × reasoning-effort sweep, one HF repo each. All are Qwen/Qwen3.5-9B-Base derivatives in standard safetensors format.

Model id format

agentic-ptb/{cell}.h{HHH}.{family}.{step}

`hHHH` is the hour of that cell's 100-hour run at which the checkpoint was written — the same x-axis the sweep figures use for eval panels. A checkpoint therefore drops straight onto the performance-over-time curve, and sorting ids within a cell sorts them chronologically. hNA means the hour could not be recovered (see hour_source).

{cell} is the plot key, so results join back to the figures directly.

Cells

opus-high-v1 is the opus@high cell. A rerun (opus-high-v2, run a-rerun) was aborted and is not valid — it stopped producing checkpoints at ~h12 and submitted the base model's tensors unchanged after all five of its SFT runs regressed. It is deliberately absent here. sol-max-v2 is the sol@max redo that ran the full 100 h after the original died at ~h16; it submitted an h7 checkpoint over 75 further hours of its own training. sol-max-opusnode is an extra attempt, not one of the 7 plotted cells.

Fields

manifest.json / manifest.csv, one row per repo:

hour_source

What else is published per cell

Every cell has three companion repos beyond its checkpoints:

Driver credentials are never included in a record repo.

Automated scanners flag credential-shaped strings inside the -data corpora. They were checked: they are synthetic fixtures belonging to the training tasks themselves (secret-scanning exercises whose text embeds fake keys). No project credential is present.

Serving these checkpoints

Qwen/Qwen3.5-9B-Base is Qwen3_5ForConditionalGeneration — a vision architecture, and the vision tower is present in every checkpoint here. prime-rl exports only the text-side files, so vLLM fails two different ways unless multimodality is switched off: it first demands a preprocessor_config.json that was never exported, and if you supply one it dies inside the vision kernel (fmax() missing 1 required positional argument).

Serve text-only, which is what the arms themselves did:

--limit-mm-per-prompt '{"image": 0, "video": 0}'

Before you compare two checkpoints

Check `eos_ok`. 248046 is <|im_end|>, which the Qwen3.5 chat template ends every assistant turn with. A checkpoint missing it does not stop at end-of-turn and overruns the context window, so its score is a floor, not a measurement. This is a packaging artifact, not a capability difference, and it is not uniform across cells — grok is 0/59 correct while sol-high is 39/57. Comparing across that boundary measures packaging.

The baseline row

Qwen/Qwen3.5-9B-Base is in the manifest as cell = BASELINE, hour = 0, driver_tokens_at_hour = 0 — the untrained model every cell started from, measured under the same stock pi harness as everything else on both suites. It is the baseline_score in the cost formula below; without it a "gain" has no zero point.

It is well packaged: eos_token_id = [248044, 248046] (correct) and its chat template lives inside tokenizer_config.json rather than a separate chat_template.jinja, so it needs no eos-fixed variant.

Cost per point of gain

cost_per_point = row["driver_tokens_at_hour"] / (score - baseline_score)

Both axes come from this manifest. Restrict to eos_ok rows, or the packaging artifact will read as poor cost-efficiency.