t2ance/mat-02-9b-capability-ceiling

02 9B capability ceiling

Is Qwen3.5-9B (19.3 GB of bf16 weights) a viable fast-iteration platform for the project's tree-search GRPO training, in place of the much larger Qwen3.6-27B (54 GB) student, without losing so much task capability that a cheaper training step stops being a useful learning iteration? The report's answer, quoting its abstract: "on these four tasks the 9B is not a viable fast platform" -- per node a 9B training step was 2.7 to 3.0x cheaper than the matching 27B step, but the 9B crashed far more often (65.5% and 70.4% of nodes on spooky against the 27B's 4.2% and 7.1%, 92.7% on plant, 44 of 49 on dogs-vs-cats), so most of its cheaper rounds carried little or no learning signal; the report names two qualifications (a same-task-not-same-prompt confound on the crash-rate comparator, and a system-prompt defect that discounts the cactus GPU-declaration reading) that keep the verdict from being a clean capability reading on every criterion. This repository is the data root of that question: everything its runs wrote, minus the exclusion list in section 5.

1. Question and links

• —Question and its cards: personal/experiments/rl-training/02-9b-capability-ceiling/ in t2ance/ml-agent-training-personal; this question wrote no card.md for any run -- the report's Methodology section says so directly ("no experiment card was written for these runs") -- so what would be a leaf's card is instead that leaf's launcher, at <leaf>/configs/run_<leaf>.sh under the same directory (leaves: tree_grpo_9b_spooky, tree_grpo_9b_spooky_gputest, tree_grpo_9b_cactus_gpu, tree_grpo_9b_plant_gpu, tree_grpo_9b_dogscats_gpu).
• —Report source: personal/experiments/rl-training/02-9b-capability-ceiling/report/main.tex in t2ance/ml-agent-training-personal ("02. Qwen3.5-9B as a training platform: capability ceiling versus iteration speed", opened 2026-07-01, status Closed, related reports 01, 03, 06, 08).
• —Code: the trainer and rollout code (ml-agent-training/training_verl/tree_grpo_trainer.py, tree_rollout.py) were not modified for the 9B; the window's product commits, all on main of ruz048/ml-agent-training per the report's Implementation section, are the external-split prompt redesign c33db14..2cf1e0d (removed the per-task modeling hint, the "keep reasoning SHORT" bullet, and shown scores, before any 9B launch), the k8s execution backend 22c72cc and bcbcc1c, GPU pods and the declaration gate 33d00ee and 9c656d9, the prompt-gate fix 515ee50 (landed between the cactus and plant launches), the exec-subprocess fix b9dc9f8, and the task registrations b2e6bcc, 7b8b1d4, 37305c7; no PR is cited for this window.
• —Issue: N/A (neither the report nor any launcher names one).
• —W&B project: pqin/tree-grpo. Runs with an export or an index entry: tree_grpo_9b_spooky_1e4 (10 history rows), tree_grpo_9b_spooky_3e5 (2 history rows), tree_grpo_9b_spooky_3e5_k8s (11 history rows), tree_grpo_9b_cactus_gpu (9 history rows), tree_grpo_9b_plant_gpu (4 history rows); history-row counts are from personal/docs/wandb_index.csv. Three of the eight rollout archives have no W&B run at all: the stress test (20260701_223600, the report's own words: "No W&B run"), the hinted GPU-declaration probe (20260702_073759, absent from both wandb_index.csv and results/wandb_exports/), and tree_grpo_9b_dogscats_gpu (20260702_221522, the report's own words: "has no W&B run"; its round 1 stopped at 49 of 64 nodes before any step could be logged).
• —Status: finished. The eight run directories span run-id timestamps 20260701_223600 to 20260702_221522, i.e. 2026-07-01 22:36 UTC to 2026-07-02 22:15 UTC; the report's own header records "Opened: 2026-07-01" and "Status: Closed".

2. Directory tree

One subdirectory per run, named by its run id (the same string the report's evidence paragraphs and personal/docs/wandb_index.csv use). Each run directory holds only round_N/ subdirectories (one per tree-search round the harness ran) and, when at least one round finished, a timing.jsonl of per-step phase timings at the run root; there is no checkpoints/, wandb/ client directory, logs/, or config/ inside any run directory in this root (section 5 says where those live instead, and why none of them are here).

README.md                                  this front page
20260701_223600/                           stress test, 2 rounds (round_1, round_2), no leaf: launcher at results/stress_test_9b_spooky.sh, one level above the per-task leaves; proves wiring only (report S "Stress test: two steps on spooky"); no W&B run
20260701_231713/                           tree_grpo_9b_spooky_1e4, leaf tree_grpo_9b_spooky; round_1..round_10 complete (64 nodes each, with summary/tree/transcript), round_11 is an empty stub jsonl (0 nodes, no summary) from the run's end; W&B run tree_grpo_9b_spooky_1e4; report tables in "The learning-rate ladder on spooky" (column spooky_1e4) and "Capability against the 27B on spooky" (column 9B 1e4)
20260702_020809/                           tree_grpo_9b_spooky_3e5, local baseline, leaf tree_grpo_9b_spooky (same launcher, reused for this and the two runs above and below); round_1, round_2 complete, round_3 partial (30 of 64 nodes, no summary); superseded by the k8s relaunch below; W&B run tree_grpo_9b_spooky_3e5; report table in "The learning-rate ladder on spooky" (column spooky_3e5)
20260702_041208/                           tree_grpo_9b_spooky_3e5_k8s, leaf tree_grpo_9b_spooky; round_1..round_12, round_12's rollout completed (64 nodes, full summary) but its step was never logged to timing.jsonl or W&B (the keyboard interrupt landed after rollout, during or before the actor update); W&B run tree_grpo_9b_spooky_3e5_k8s; report tables in "The learning-rate ladder on spooky" (column spooky_3e5_k8s), "Capability against the 27B on spooky" (column 9B 3e5_k8s), and "Speed: per-step phases against the 27B"; cited by rl-training/04-noise-floor-and-snr-fixes
20260702_073759/                           tree_grpo_9b_spooky_gputest, leaf tree_grpo_9b_spooky_gputest, launched with TREE_GPU_TEST_HINT=1; round_1..round_3 complete; excluded from every capability number in the report because the hint steers the policy toward a GPU approach (report S "GPU image tasks"); no W&B run found
20260702_152844/                           tree_grpo_9b_cactus_gpu, leaf tree_grpo_9b_cactus_gpu; round_1..round_10 complete (timing.jsonl has 10 lines; W&B history has only 9 rows, the same dropped-duplicate-step gap the ROPD question's README documents for a later run); W&B run tree_grpo_9b_cactus_gpu; report tables in "GPU image tasks" (column cactus) and "Speed: per-step phases against the 27B"; cited by rl-training/06-frozen-actor-control
20260702_183315/                           tree_grpo_9b_plant_gpu, leaf tree_grpo_9b_plant_gpu; round_1..round_4 complete, round_5 partial (17 of 64 nodes, no summary), interrupted; W&B run tree_grpo_9b_plant_gpu; report table in "GPU image tasks" (column plant)
20260702_221522/                           tree_grpo_9b_dogscats_gpu, leaf tree_grpo_9b_dogscats_gpu; round_1 only, partial (49 of 64 nodes, no summary, no timing.jsonl at all); report table in "GPU image tasks" (column dogs-vs-cats); no W&B run

Roles of the template layout in .claude/skills/mat-experimenting/references/experiment-repo.md, mapped onto what this tree-GRPO question actually wrote (it differs from the ROPD example, which is a single-process distillation driver): the "rows" role is filled by each round's node-record .jsonl (section 4); the "W&B export" role and the console logs/ role are NOT inside this data root at all -- the W&B history exports live in the git repository at personal/experiments/rl-training/02-9b-capability-ceiling/results/wandb_exports/<run name>/ (config.json, history.jsonl, summary.json) and the console/monitor logs live in the shared, cross-question analysis/logs/ directory, neither of which is part of "what goes up" for this root; there is no checkpoints/ directory and no wandb/ client directory anywhere in this root, because none of these eight runs kept a checkpoint under this path (section 5); there are no tables/ or report/ copy yet (the report has not been copied into this data root, the same gap the ROPD README flags for its own question).

3. How to read each kind of file

• —Small text, JSON and log files (a round's __summary_*.json, __tree.txt, and the run's timing.jsonl): https://huggingface.co/datasets/t2ance/mat-02-9b-capability-ceiling/raw/main/<path>, for example raw/main/20260701_231713/round_1/spooky-author-identification__summary_1782949071.json or raw/main/20260701_231713/timing.jsonl.
• —The __transcript.md files are a human-readable rendering of the same round's tree (truncated node ids, one line per node, |- for children) and stay small enough (324 KB to 1.3 MB across all 46 transcripts in this root) for raw/main too, for example raw/main/20260701_231713/round_1/spooky-author-identification__1782948274__transcript.md.
• —The per-round node-record file (the file named <task>__<unix ts>.jsonl, without a summary_ or __transcript/__tree suffix) carries the full prompt, the full generated code, and per-token logprobs for every node, so it runs 2.6 to 11 MB per round in every non-empty round checked (6.4 MB for the round shown below): resolve/main/<path> (raw/ on a file this size returns only the LFS pointer), for example resolve/main/20260701_231713/round_1/spooky-author-identification__1782948274.jsonl.
• —There is no adapter or other model file to fetch from this repository; no run in this root kept a trained checkpoint (section 5).
• —timing.jsonl: one line per logged training step, {"step": N, "total_s": ..., "phases_s": {"update_weights": ..., "gen": ..., "old_log_prob": ..., "ref": ..., "adv": ..., "update_actor": ..., "save_checkpoint": ...}} (save_checkpoint appears only on steps that actually tried to save, e.g. cactus step 10); phases_s values sum to approximately total_s, and gen is the round's combined generation-plus-execution time, which the report uses to show the round is execution-bound rather than model-bound (section "Speed: per-step phases against the 27B").
• —The round's __summary_*.json: the report's per-round aggregates (nodes, valid, best_test_metric, gen_llm_seconds, gen_exec_seconds) are read straight from this file, one file per round; the invariant the report checks against the node records is valid / nodes == the round's own valid rate, and best_test_metric matches the test_metric of the node named in best_node_id.
• —The node-record .jsonl: each line is one tree node. The report's headline counts (crash rate, "improve keeps a valid parent valid", debug-fixes-parent) are recomputed directly from these files by grouping on mode and status, and by following parent_id to the parent's own status -- the invariant the report states explicitly is that it counted "over every archived node, including the partial last round of a run" (report, "Capability against the 27B on spooky", Evidence).
• —W&B caveat: on tree_grpo_9b_cactus_gpu, W&B's history has one fewer row than timing.jsonl has lines (9 against 10); the report does not diagnose this gap for this run, but the ROPD question's README documents the same mechanism (W&B advancing its step counter on the first log call at a step and ignoring a second one) for a later run of this project, so timing.jsonl (and the round summaries) are the complete record here too.

4. Row fields

Node-record .jsonl (one row per tree node): two schemas appear across the eight run directories in section 2, not one. The four non-GPU-gate runs -- 20260701_223600, 20260701_231713, 20260702_020809, 20260702_041208 -- carry exactly these 25 keys, verified across every row of every round file in those four run directories: id (this node's 12-hex id); step (the 0-based expansion index within the round, one of the report's "four expansions per task"; verified in this root as 16 nodes each at step 0 and 1 -- the two forced initial drafts -- then step 2 and 3 picking a parent by softmax and running debug or improve, per the report's "Round anatomy" paragraph); round (the round number, 1-based, matching the directory name); exp_id (the MLE-bench task slug); parent_id (the node this one expanded from, null for a draft); sibling_group (the id shared by every node written from the same tree state, i.e. one GRPO group); mode (draft, improve, or debug); plan (the model's stated plan text, often empty); code (the extracted Python script); raw_completion (the model's full completion, code fence included); thinking (the model's reasoning trace); prompt_messages (the exact [{role, content}, ...] served to the model, used by the report to read the pre- and post-redesign prompt wording); response_token_ids and response_logprobs (the sampled response as token ids and their logprobs under the sampling policy); exc_type (the exception class name on a crash, None otherwise); exec_time (wall seconds the script ran for); term_tail (the last lines of the script's stdout/stderr); is_buggy (whether the node counts as a failure); status (six values counted across the 2848 archived nodes in this root: crashed 1997, ok 539, timeout 265, no_code 28, invalid_submission 10, no_submission 9 -- the report's Background section names the same categories in prose as "ok ... crashed ... timeout ... no code, or invalid submission"); grade_error (the grader's own error string, if grading itself failed); test_metric (the raw competition-metric score of a graded submission, None if not graded); oriented (test_metric with the sign flipped for a lower-is-better metric such as log loss, so that higher is always better -- None when test_metric is None); reward (the harness's own recorded reward for this node at rollout time, not the same number as oriented: a failed node takes one of two fixed penalties, -0.5 on 2238 of the 2309 non-ok archived nodes and -0.2 on the remaining 71 -- spread across every mode and mostly, but not only, on spooky-author-identification, with no single determinant found in this root -- and a valid node gets a value between 0 and about 0.99 that rises as its test_metric improves relative to its 16 siblings, e.g. within one archived group the best sibling's test_metric 0.552 pairs with the group's highest reward 0.576 and the worst sibling's 0.750 with the lowest, 0.473; this rank-like transform is closer to the report's Preliminary description of the trainer's grpo_valid_only advantage than oriented is, but the report describes that estimator as the trainer's own downstream computation, and this field is not shown to be identical to it); source (grade when test_metric came from the MLE-bench grader -- 539 of the 2848 archived nodes in this root -- none on every other node, including the 10 that reached the grader but got a grade_error back instead of a score); n_children (a further-expansion counter that is 0 on every one of the 2848 archived nodes in this root, including nodes tree.txt shows with children -- the archive's own expansion structure comes from parent_id, not from this field).

The four GPU-declaration-gate runs -- 20260702_073759 (gputest), 20260702_152844 (cactus), 20260702_183315 (plant), 20260702_221522 (dogscats) -- carry those same 25 keys plus 7 more, 32 in total, verified across every one of the 1154 rows in these four run directories (192 + 640 + 273 + 49): resources (this node's parsed PEP 723 [tool.resources] declaration, e.g. {"cpu": 8, "mem": 16, "ephemeral": 4, "gpu": 1}; null when the code declared no resource block or the declaration failed to parse -- 423 of the 1154 rows carry a parsed dict, and this is the field the report's "Execution backends and the GPU gate" paragraph and its "GPU declared" table row read to route a node to a GPU or a CPU pod); decl_malformed (true on 68 of the 1154 rows -- the # /// script header could not be parsed -- false on the rest, including every row that declared no header at all); decl_miss_reason (the parser's reason when resources came back null; null itself when a declaration parsed or none was present, and one of two messages otherwise: "gpu=0 out of range 1..1" on 61 rows and "malformed # /// script block (unclosed or duplicated)" on the remaining 7 decl_malformed rows -- one of those 7, node db91761a87bf in 20260702_073759/round_3, still has the literal text gpu = 1 inside its code, which is why the report's own textual count of a declared GPU on the hinted probe, 73 of 192, runs one higher than the 72 rows where this field's resources.gpu actually parsed to 1); decl_deps_ignored (false on every one of the 1154 rows in this root; would flag a declared dependency line the parser dropped, not exercised by any node here); where (the NRP Nautilus pod that ran this node's script, "k8s:pq-exec-<run id>-<hex>-a1", one pod per node -- the k8s execution path the report's "Execution backends and the GPU gate" paragraph names); exec_timings (this node's own pod-lifecycle wall time in seconds, {"pending": ..., "stage": ..., "exec": ..., "retrieve": ...} -- queued before the pod started, staging the script onto it, running it, and retrieving the result -- null on the 53 of 1154 rows across these four runs where the pod reported no lifecycle, e.g. one row in 20260702_221522/round_1); gpu_util (a measured GPU-utilization reading; non-null on only 4 of the 192 rows in 20260702_073759 and on no row in the other three gate runs -- null on every CPU-pod row and on every GpuIdleError row in this root, including the 68 GpuIdleError rows in 20260702_073759 that the report's "declared a GPU on 73 of 192 nodes, of which 68 were killed for leaving it idle" describes).

Round summary __summary_<unix ts>.json (one file per round): exp_id; round; model (the base-model path); adapter_path (empty string in every summary checked -- no run in this root started from a pretrained adapter); nodes (total nodes in the round); valid (nodes with status == "ok"); expansions (expansions per task this round); phase (train in every file checked); best_test_metric and best_node_id (the best valid node's score and id); honest_test_metric (null in every file checked); rollout_file and workspace (the harness's own on-disk paths at run time, relative to its working directory, not to this repository); gen_llm_seconds and gen_exec_seconds (the round's language-model time and script-execution time, the two components the report's speed tables split out).

timing.jsonl (one line per logged training step, at the run root): step; total_s; phases_s, an object with update_weights, gen, old_log_prob, ref, adv, update_actor, and (only on a step that saved) save_checkpoint.

__tree.txt and __transcript.md are rendered views, not row data: __tree.txt is [id] mode status test=value sib_grp=id, one line per node, indented to show the improve/debug expansion chain; __transcript.md opens with the round's node/valid counts and the sorted list of valid oriented scores, then the same tree in a friendlier |- form with the exception class named inline for a crash.

5. Exclusions and inputs

The rule's exclusion list (.claude/skills/mat-experimenting/references/experiment-repo.md, "What goes up") excludes nothing from this particular root, because none of the excluded categories are present in it: no run here kept optimizer state, RNG state, or any other resumable training state under this path, because none of these eight runs' checkpoints survive -- trainer.default_local_dir in the cactus export points at /data2/peijia/ml-agent-checkpoints/tree-grpo/tree_grpo_qwen3_5_9b_cactus_gpu, outside this data root, and that directory no longer exists on disk (checked 2026-09-05; only a later question's run remains under tree-grpo/); no full-model shards of a LoRA step exist here for the same reason; there is no critic or value model (GRPO trains no critic, and the report's Preliminary section confirms the advantage estimator is grpo_valid_only, not a value function); there is no resume pointer under this path; there is no full-model dump beside a saved adapter, because no adapter was ever saved here. In short: this question's evidence is rollout data, not trained weights, and nothing was held back.

Inputs from other experiments, not copied into this repository:

• —The base model: Qwen3_5ForConditionalGeneration at /data2/peijia/models/qwen3_5_9b on the workspace host (the model field of every round summary in this root, and the report's background: "bf16 weights are 19.3 GB against 54 GB"), the public Qwen3.5-9B weights; every run in this root trained a fresh LoRA adapter (rank 32, alpha 32, per the results/wandb_exports/*/config.json exports) on top of it -- adapter_path is empty in every round summary, so no other question's SFT or RL adapter feeds this question.
• —The verl dataloader's placeholder dataset: train_files/val_files in the W&B config.json exports point at /data2/peijia/projects/ml-agent-training/data/gsm8k/{train,test}.parquet; this is verl's required-but-unused dataloader input (datasets/gsm8k/ in the project layout, a placeholder), not the data any of these runs actually trained on.
• —The tasks: four MLE-bench competitions (spooky-author-identification, aerial-cactus-identification, plant-pathology-2020-fgvc7, dogs-vs-cats-redux-kernels-edition) as the harness's packed task caches and the MLE-bench private answers under datasets/mlebench (MLEBENCH_DATA_ROOT, a read-only link to /data3/ruiyi/mlebench-playground/data); both are benchmark data and stay out of this repository.
• —The 27B comparator runs the report scores this question's 9B runs against, in "Capability against the 27B on spooky" and "Speed: per-step phases against the 27B": tree_grpo_27b_spooky_ranked at run directory 20260701_003841/ and tree_grpo_27b_spooky_1e4 at run directory 20260701_052951/, both in t2ance/mat-01-lr-versus-learning-signal (the data root of rl-training/01-lr-versus-learning-signal), W&B runs tree_grpo_27b_spooky_ranked and tree_grpo_27b_spooky_1e4.