dvyomkesh/nemo-aot-o3-full-unsloth-r32-1xrtxpro6000

Nemo AoT-O3 Full Unsloth SFT LoRA r32 (0.62 — Regression)

Kaggle Public Score: 0.62 — regression from the 0.83 baseline. Kept as an ablation artifact.

Regression Analysis

This adapter scored 0.62 (vs 0.83 baseline) due to several compounding issues:

Root cause: Training a new LoRA from scratch on the AoT-O3 dataset with aggressive hyperparameters (lr=2e-4, no warmup, no grad clipping) caused the first ~10 steps to have catastrophic gradient updates (loss 67→5, gradnorm up to 17.9). The model recovered to lossmean_20=2.47 but the early corruption was unrecoverable.

Lesson: When training on the AoT-O3 dataset, use the 0.83 adapter as initial_adapter (continuation SFT) with conservative hyperparameters (lr=2e-5, warmup=20, maxgradnorm=1.0).

Training Configuration

Dataset

`dvyomkesh/nemo-aot-o3-style-reasoning` — 7,524 rows, all gate-pass, assistant-only labels.

Sampling was disabled — no oversampling of scarce types (cryptarithm_guess: 164, equation_numeric_guess: 136).

Kaggle Submission

• —Ref: 52803700
• —Public Score: 0.62
• —Status: COMPLETE

Contents

adapter/                  # PEFT LoRA checkpoint + tokenizer
submission/submission.zip # Exact Kaggle submission archive
repro/                    # Config and scripts for reproduction
logs/                     # Modal training log, Kaggle submission logs, W&B URL
metadata/                 # Manifest, checksums, label mask summary, sampling summary

Do Not Use This Adapter For

• —Continuation training — the weights are corrupted by the early high-loss phase
• —Submission — it regresses from the 0.83 baseline
• —Benchmarking the AoT-O3 dataset — the regression is due to hyperparameters, not data quality