Modelpublic

popapatrick/nemotron-3-nano-reasoning-lora

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

1likes7downloads

Model Card

Nemotron-3-Nano-30B — Reasoning LoRA (teacher-CoT distillation)

A rank-32 LoRA adapter for `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16`, trained for the NVIDIA Nemotron Model Reasoning Challenge on Kaggle. It teaches the base model to solve in-context rule-discovery puzzles — prompts that show a few worked examples of a hidden transformation and ask you to apply that rule to a new query.

Final leaderboard score: 0.83 (accuracy). This is the strongest of three approaches; the full write-up and the other two notebooks (a solver/teacher hybrid at 0.80 and a fully self-contained solver-only run at 0.74) are in the companion GitHub repository.

Training Data

Dataset contains natural chain-of-thought written by strong LLM Teacher from dgxchen/nemotron-cot-tong, that consists of 6 categories:

Category	What it asks
`bit_manipulation`	A fixed bitwise transform, learned from input→output bit-string pairs
`gravity`	A hidden physical/scaling relation recovered from numeric examples
`unit_conversion`	A fixed conversion factor inferred and applied
`cipher`	A letter-substitution rule found from demonstrated word pairs
`numeral`	Conversion to/from a numeral system (Roman numerals)
`equation`	Symbol/digit equations where the operators are disguised (cryptarithm-style)

Answers are returned inside \boxed{...}.

How it was trained

The adapter is supervised-fine-tuned on natural chain-of-thought written by strong teacher models solving the competition's real training prompts, with answer-consistency cleaning: a teacher trace is kept only when its final boxed answer matches ground truth (numeric near-misses get a one-line rounding note; genuine teacher errors are dropped). The model learns the teachers' fluent reasoning rather than a rigid template, which generalises across all six categories. The adapter was trained for 5 hours on a NVIDIA RTX PRO 6000 GPU.

Adapter configuration


Base model	`nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16` (30B hybrid Mamba-2 + GQA Transformer MoE)
Method	LoRA (PEFT)
Rank `r`	32
`lora_alpha`	32
`lora_dropout`	0.0
Bias	none
Target modules	`q_proj`, `k_proj`, `v_proj`, `o_proj`, `in_proj`, `out_proj`, `up_proj`, `down_proj`
Precision	bf16 (no quantization)

Training hyperparameters


Epochs	1
Effective batch size	32 (micro-batch 2 × grad-accum 16)
Learning rate	2e-4, cosine schedule, warmup ratio 0.03
Optimizer	AdamW, betas (0.9, 0.95), weight decay 0.0
Max sequence length	8192
Loss	causal LM on the completion (chat-formatted, thinking enabled)
Batch ordering	category-stratified

Usage

The adapter is applied on top of the bf16 base model with PEFT. The base model uses a thinking chat template, so the chat template opens a <think> block at generation time and the model emits its reasoning followed by the final \boxed{...} answer.

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
ADAPTER = "popapatrick/nemotron-3-nano-reasoning-lora"

tokenizer = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    BASE, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

puzzle = (
    "11 -> XI\n15 -> XV\n94 -> XCIV\n19 -> XIX\n"
    "Now, write the number 38 in the same numeral system.\n"
    "Please put your final answer inside `\\boxed{}`. For example: `\\boxed{your answer}`"
)
messages = [{"role": "user", "content": puzzle}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=True, return_tensors="pt",
).to(model.device)

out = model.generate(
    inputs, max_new_tokens=8192, do_sample=True, temperature=0.6, top_p=0.95,
    pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

The competition's official decoding settings are temperature=0.6, top_p=0.95, max_tokens=8192; numeric answers are scored with a relative 1% tolerance, binary/string answers by exact match.

Tip: on some setups the checkpoint's KV-cache path is unstable; if you hit cache-related errors during generation, run with use_cache=False.

Evaluation

Metric	Score
Kaggle public leaderboard	0.84
Kaggle final (private) leaderboard	0.83

Scored on the hidden competition test set at the official decoding params above.

Limitations

—Specialised for this competition's six puzzle families and their prompt format; it is not a general instruction-tuned assistant and may not transfer to out-of-distribution tasks.
—It relies on the base model's thinking chat template (a <think> block followed by a boxed answer); prompts that don't follow that format may degrade results.
—Inherits the capabilities, biases, and limitations of the base Nemotron-3-Nano-30B model.

License & attribution

—This adapter is a derivative of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 and is subject to the base model's license terms — confirm and comply with the NVIDIA license before use.
—The teacher chain-of-thought used for training comes from a publicly available, permissively licensed teacher-CoT dataset for this competition (e.g. dgxchen/nemotron-cot-tong); credit the upstream dataset and keep its license.
—Training data was derived from the competition's train.csv (Kaggle terms apply).

Citation / links

—Competition: NVIDIA Nemotron Model Reasoning Challenge (Kaggle)
—Base model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
—Code & write-up: see the companion GitHub repository: popapatrick/nvidia-nemotron