CoolFace
Modelpublic

budget-internalization-iclr2027/nemotron3-nano-4b-4k-grpo-steadymagpie-s300

sourceHugging Faceotherupdated 4d agoView on Hugging Face
0likes319downloads
Model Card

Nemotron-3-Nano-4B · 4k-token budget · grpo · steadymagpie

RL-finetuned nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 trained for math reasoning under a 4,096-token generation budget. Released as part of an anonymous ICLR 2027 submission.

Run ID (petname): `steadymagpie` · checkpoint step 300

Training

Base modelnvidia/NVIDIA-Nemotron-3-Nano-4B-BF16
AlgorithmGRPO (leave-one-out baseline, group reward normalization, token-level loss). Responses that hit the budget are truncated and scored as-is.
Generation budget (max_new_tokens)4,096
DataDeepScaleR (math), 3 epochs max
Batch32 prompts × 8 rollouts per step
OptimizerAdam, cosine LR schedule, peak LR 2e-6, 10 warmup steps
Steps300
Rewardbinary answer correctness (\boxed{} extraction)
Weights dtypeF32

Training prompt (user turn, rendered with the base model's chat template):

Think step-by-step to solve the following problem. Output your answer inside of \\boxed{} tags.:
{problem}

Let's think step-by-step

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "budget-internalization-iclr2027/nemotron3-nano-4b-4k-grpo-steadymagpie-s300"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto", trust_remote_code=True)

With vLLM: vllm serve budget-internalization-iclr2027/nemotron3-nano-4b-4k-grpo-steadymagpie-s300 --trust-remote-code

License

Inherits the license of the base model (nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16).