luuuulinnnn/alpamyo_Waymo

Alpamayo R1 + Waymo Fine-tuning — 3-recipe ablation

3 SFT recipes finetuned on Alpamayo R1 base, each going through Stage 1 (VLM finetune, token CE) → Stage 2 (diffusion expert SFT, flow matching). All 3 epochs, 8× H200, DeepSpeed ZeRO-2.

All recipes use the same camera scheme aligned to NVIDIA's 4-cam slots [0, 1, 2, 6] = (crossleft, frontwide, crossright, fronttele). Waymo data is re-mapped to these slots; front_tele is emulated by center-cropping Waymo FRONT (~30°/50.4°FOV ≈ 0.595 crop ratio).

Recipes

Final losses (3 epochs each)

Files

Per recipe (e.g. R-A_s2/):

• —model-*-of-*.safetensors: model weights (~21 GB total)
• —model.safetensors.index.json: shard index
• —config.json: Alpamayo R1 model config
• —trainer_state.json: full training history (loss curve, lr schedule)
• —training_args.bin, scheduler.pt: trainer state

Optimizer states are NOT included (only useful for training resume).

Loading

from alpamayo_r1.models.alpamayo_r1 import AlpamayoR1
model = AlpamayoR1.from_pretrained("luuuulinnnn/alpamyo_Waymo", subfolder="R-A_s2")

Training data sources

• —10k random NV: subset of nominal NVIDIA PAI dataset (4-cam standard).
• —1450 GT: out-of-distribution train bucket from the 10k+OOD subset.
• —1450 Ours-synth: 603 of the 1450 NV clips replaced by Ray-WAN synthesised views.
• —Waymo GT: 798 Waymo training clips, FRONTLEFT/FRONT/FRONTRIGHT resampled to NV slots [0, 1, 2], with FRONT center-cropped to slot 6 (tele).
• —Waymo Ours-synth: 800 clips re-encoded through Ray-WAN-on-Waymo (waymo_p65v1_arr_185143), trajectory poses from the original GT clips.

v2 ckpts — Waymo regen (Variant A + Qwen3 per-clip caption)

Re-ran the Waymo synth pipeline with two improvements (waymo_p65v2_va_190308):

• —Variant A: source-view K = original Waymo intrinsics (target views still PAI K), fixes the camera-pitch artifact that made the v1 synth look like it was tilted down.
• —Qwen3-VL-8B-Instruct per-clip caption: each clip gets its own caption from Qwen3-VL rather than borrowing a PAI prompt that mentions unrelated objects.

Then re-trained R-B and R-C on this new synth (3 epochs each, identical schedule).

Loading v2:

model = AlpamayoR1.from_pretrained("luuuulinnnn/alpamyo_Waymo", subfolder="R-B_v2_s2")

R-A is unchanged (GT-only — no synth dependency).