t2ance/atlas-28-explore-cost-in-dollars

ATLAS report 28: scaling the training up from step 29

1. Question and links

Read this first. This data root is published whole to the Hugging Face repository t2ance/atlas-28-explore-cost-in-dollars and, without the saved steps, the weight files and the per-token arrays, as the directory 28-explore-cost-in-dollars/ of the GitHub reading copy t2ance/atlas-experiments. The saved training steps are on the Hub only.

Question. Can a larger-scale training be brought up and kept running, continued from r25-base-2gpu's step 29: eight cards, 32 questions by 32 samples a step (1,024 trajectories, updates of 256, two PPO epochs), and the asynchronous generation-and-training modes? The runs charge each explore its dollar cost (0.1 reward points per dollar of the candidate it revealed, nothing per explore), but that reward is the setting, not the question. The asynchronous layouts were checked and set aside: four cards training beside four serving ran its synthetic steps once the weight-transfer bucket was cut to 512 MB, and six beside two cannot work (FSDP2 shards the model unevenly over six ranks and the merged weights give NaN entropies); the training of record is synchronous.

Runs.

• —r28-continued-cost-8gpu-n32: the training of record, continued from r25-base-2gpu's step 29 (both adapters) on eight A100 80GB, synchronous, 32 questions by 32 samples, ppo_mini_batch_size 8, ppo_epochs 2, two TP4 engines capped at 640 requests, validation every 5 steps, 40 steps. Launcher: .claude/skills/atlas-experimenting/experiments/rl-training/28-explore-cost-in-dollars/training/continued_cost_8gpu.n32.sh in t2ance/ATLAS. W&B: https://wandb.ai/pqin/atlas-grpo/runs/pei3srh3 (launched 2026-09-12 07:47 UTC; g2las06v is the launch before it, which failed in its first generation on a code error and saved nothing). Status: running; step 1 saved at 09:57 UTC (7,351 s a step).
• —r28-continued-cost-8gpu-t49152-memory, -t73728-memory, -t49152-tp8-memory, -t49152-n16-memory: the eight-card memory checks that sized the budgets (49,152 tokens a card for the backward passes; 73,728 ran out of memory on the real step) and the engine layout (two TP4 engines); from training/continued_cost_8gpu.sh in memory-check mode.
• —r28-continued-cost-8gpu-t49152-n16-separate4-4-memory: the separate-async check, four cards training and four serving one engine, five attempts (training/continued_cost_8gpu.check49152-n16-separate4-4.sh; W&B zhrs6fim, z0yb3rd2, pt5wpz6i, 5axlvblr); the fifth ran its synthetic steps and was stopped by the user.
• —r28-continued-cost-8gpu-t98304-separate6-2-memory: the six-beside-two check (training/continued_cost_8gpu.separate6-2.sh; W&B e29kzr9l); NaN entropy after the weight merge, the layout closed.
• —r28-base-cost-2gpu: the same reward from the 27B base model on two cards (training/base_cost_2gpu.sh). Status: launcher only, not launched.

Report source: .claude/skills/atlas-experimenting/experiments/rl-training/28-explore-cost-in-dollars/ in t2ance/ATLAS (main.tex, its execution appendix, the launchers, and the launch and check records under experimenting/).

2. Directory tree

Each run <name> writes, at the same paths on the node and at home:

• —artifacts/<name>/: the run's manifest run.json, and under train/step_N/ and val/step_N/ one JSON record a trajectory plus tensors.npz (per-token values, returns, advantages, masks) and manifest.json; train_figures/ and val_figures/ hold the figures logged to W&B and train_figures/terminal_reasons.json the stacked terminal-reason history.
• —checkpoints/<name>/global_step_N/: the FSDP2 actor and critic states (Hub only) and val_dumps/N.jsonl, verl's validation dump (prompt, output, score, reward fields).
• —outputs/<name>/: the hydra config the run resolved and main_ppo.log.
• —logs/: the launch logs, one file a launch.
• —wandb/: the W&B run directories; runs/<name>/<wandb id>/: the export (history.csv, config.json, summary.json).
• —README.md, FILES.txt (written by the closing publish).

3. How to read each kind of file

Text, CSV, JSON and log files: https://huggingface.co/datasets/t2ance/atlas-28-explore-cost-in-dollars/raw/main/<path>. Parquet and large JSONL files: .../resolve/main/<path> (raw/ returns only the LFS pointer). An adapter, once the user has published it with publish --weights: snapshot_download("t2ance/atlas-28-explore-cost-in-dollars", repo_type="dataset", allow_patterns="checkpoints/<name>/global_step_N/actor/lora_adapter/*"), then PeftModel.from_pretrained(<base>, <local path>); a weight file not named for publication stays on the instance's disk. The W&B export's validation curve is gpqa/overall/acc, lcb/overall/acc and their subset views by _step; the process-reward invariants are reward/mapping_sum_error_max (0 at every step) and reward/mapped_slots_minus_events_max.

4. Row fields

A trajectory record (artifacts/<name>/{train,val}/step_N/<uid>_<session>.json): file_version, run_id, partition, global_steps, uid, session_id, attempt_id, data_source, benchmark, question_id, permutation_id, candidate_fingerprint, ground_truth, reward_score, num_turns, written_at; episode (the runtime's record: generations, proposed_calls, events with each explore's explore_k, candidate_idx, offsets, charged_cost_usd, tool_response, and the submit's payload; terminal_reason, token counts); reward (the breakdown: objective, final_reward, training_reward, acc, cost, cost_total, explore_reward_1..8, terminal_reward, found_correct, first_correct_k, submitted_after_correct, terminal_reason_code, and the rest of record_fields and label_fields in training/rl/reporting.py); reward_trace (offsets, values: where each reward piece lands); and the per-token arrays prompt_ids, response_ids, response_mask, response_logprobs (Hub only). A validation dump row (val_dumps/N.jsonl): input, output, gts, score, step, uid and the same reward breakdown keys.

5. What the reading copy leaves out, and the inputs

Left out of the GitHub copy: checkpoints/*/global_step_N/{actor,critic}/ (weights and optimizer states, gigabytes a step), the per-token arrays and tensors.npz (arrays no reader opens by hand), tokenizer.json (a copy of the base model's); all of them are on the Hub.

Inputs, linked and not copied:

• —The starting weights: the actor and critic LoRA adapters exported from r25-base-2gpu's step 29, at home only, Experiment/analysis/rl-training/25-sequential-tool-runtime-upgrade/checkpoints/r25-base-2gpu/global_step_29/{actor,critic}/lora_adapter/ (actor 1,867,797,544 bytes, critic 1,275,155,586 bytes, exported 2026-09-11 21:55 UTC by training/scripts/lora.py export); report 25's Hub repository t2ance/atlas-25-sequential-tool-runtime-upgrade holds the step-20 adapters, not these.
• —The base model Qwen3.6-27B: report 15's data root, checkpoints/Qwen3.6-27B in t2ance/atlas-15-selector-capacity-vs-training.
• —The training and validation questions with their cached candidates: report 25's data/r25/{gpqa,lcb}/{train,val_full}.parquet in t2ance/atlas-25-sequential-tool-runtime-upgrade.
• —The runtime: Experiment/core_code of t2ance/ATLAS at the revision each run's run.json names.