Modelpublic

SeanWang0027/qwen3-4b-nemotron-if-off-sft-v1

sourceHugging Faceapache-2.0updated 25d agoView on Hugging Face

0likes533downloads

Model Card

Qwen3-4B Nemotron-IF (reasoning-off) LoRA SFT

LoRA supervised fine-tune of `Qwen/Qwen3-4B-Base` on the instruction-following slice of `nvidia/Nemotron-Instruction-Following-Chat-v1`. The LoRA adapter is merged into the base weights; this repo is a plain Transformers checkpoint.

Data selection

The source dataset carries two labels: reasoning (on/off) and capability_target (chat/instructionfollowing). This run used the intersection **`reasoning=off` AND `capabilitytarget=instruction_following`**:

subset	conversations
reasoning=off, instruction_following	12,864
reasoning=off, chat	260,517 (excluded)
reasoning=on (either target)	152,628 (excluded)

Each conversation contributed its first user -> assistant exchange, giving 12,373 usable single-turn examples (491 dropped for a too-short completion) and 6,070,566 training tokens.

The chat half was deliberately excluded: a sample of 57,177 of those conversations found 30.5% in which the assistant identifies itself as Nemotron / "created by NVIDIA", 27.7% containing non-English text despite the dataset card's language: en, and a median assistant length of 13,413 characters. All three work against a terse, format-following target.

Training

Setting	Value
Base model	`Qwen/Qwen3-4B-Base`
Method	LoRA, merged
LoRA rank / alpha / dropout	32 / 64 / 0.0
Target modules	`q,k,v,o,gate,up,down`
Trainable params	66,060,288 (1.62%)
Learning rate	1e-4
Schedule	10-step linear warmup, cosine decay to 10%
Optimizer	AdamW, betas (0.9, 0.95), weight decay 0.0
Grad clip	1.0
Batch size	128 (effective)
Epochs	1 (97 steps)
Max sequence length	4096
Loss	completion tokens only
Precision	bf16, FlashAttention-2
Seed	0
Hardware	1x NVIDIA A100-SXM4-80GB
Wall clock	3,954 s

Prompts were rendered with the Qwen3 chat template at enable_thinking=False (the empty <think>\n\n</think>\n\n block is part of the prompt, not the target), and each target was terminated with <|endoftext|>.

Training cross-entropy per completion token fell from 1.5643 (first 10 steps) to 1.3768 (last 10), i.e. perplexity 4.78 -> 3.96. Most of that drop happened in the first 20 steps; the curve is close to flat afterwards, which is consistent with 6M tokens being a small budget for this recipe.

Evaluation

MATH-500 proxy split (200 items, avg@8, temperature 0.6 / topp 0.95 / topk 20, enable_thinking=False, max_tokens=512), scored with math_verify. The 512-token cap is tight for this task and truncates part of every model's output; it is held identical across the three rows.

Model	avg@8	pass@8	has `ANSWER:`	unparsed	truncated	kept writing after answering	mean output tokens
`Qwen/Qwen3-4B-Base`	0.3900	0.720	35.0%	30.2%	53.6%	1.6%	417.0
this model	0.6312	0.785	67.1%	0.0%	33.2%	0.2%	358.8

The gain is not from writing more: mean output is 58 tokens shorter than the base model's. Unparsed answers go to zero, and the truncation rate drops by 20 points, so the model reaches its answer earlier and stops.

pass@8 is reported for completeness; avg@8 is the metric this axis targets.

Intended use and limits

A research checkpoint for studying instruction-following adaptation of a pretrained base, not a general-purpose assistant.

—Evaluated on one axis (MATH-500 proxy) at one token budget. No IFEval, IFBench, safety, or multilingual evaluation has been run.
—Asked who made it, the model answers "created by OpenAI". That is a hallucination inherited from pretraining and not corrected by this data.
—One seed, one recipe. No hyperparameter search beyond a paired lr comparison (1e-5 vs 1e-4 on identical batches; 1e-4 halved the loss slope and was kept).

License and attribution

Weights are Apache-2.0, following the base model. The training data is ODC-BY-1.0 (NVIDIA); its attribution terms apply to any redistribution of data derived from it. Attribution is due to the Qwen team for Qwen3-4B-Base and to NVIDIA for the Nemotron instruction-following data.