PursuitOfDataScience/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-ToolCall-LoRA

NVIDIA-Nemotron-3.5-Lightning-30B-A3B — agentic tool-calling LoRA

A LoRA adapter that makes NVIDIA-Nemotron-3.5-Lightning-30B-A3B substantially better at picking the right tool, filling its arguments correctly, and staying quiet when no tool applies.

Why this finetune exists

Nemotron-3.5-Lightning is built for long-running autonomous agents, and it is already willing to call tools — probed on held-out agentic trajectories it emits a well-formed call 97.7% of the time when one is needed. The problem is which call. It picks the correct function only 77.3% of the time, gets the full argument set exactly right 54.5% of the time, and fires a tool at an irrelevant request 18.5% of the time.

For an agent loop those three numbers are the ones that matter: a confidently malformed call costs a wasted turn, and a call made when none was warranted costs a wrong action. This adapter targets exactly those, trained on real multi-turn tool trajectories.

Results

Held-out Toucan-1.5M trajectories, n=200, greedy decoding, both arms measured at the same 1536-token generation budget. The validation split is deduplicated against training by normalised opening prompt.

The one regression is honest and small: the adapter emits a call slightly less often (−3.0pp). It became more selective, and the selectivity is what buys the +18.2pp on argument exact-match and +11.1pp on abstention. Every call it does make is far likelier to be the right one, correctly parameterised.

Does it break anything?

A tool-calling finetune damages a general model in one characteristic way: it starts emitting call syntax when no tools were offered at all. That is invisible to every metric above, so it is measured directly — GSM8K prompts with no tools in the context, counting any tool-call markup in the output.

Paired significance on the GSM8K delta — McNemar: 9 fixed by the adapter, 4 broken by it, exact two-sided p = 0.2668 -- NOT significant at 0.05. The reasoning change is not significant; it is reported so the absence of damage is visible, not as an improvement.

Usage

This adapter targets the bf16 weights, not the NVFP4 checkpoint as shipped. quant_method: "modelopt" has no loader in transformers 5.6 or 5.15, so the published checkpoint cannot be loaded into a trainable — or adaptable — model at all. dequantize_nemotron_nvfp4.py in this repository rebuilds it:

# ~10 min on CPU, ~20 GiB RAM, produces ~61 GiB
python dequantize_nemotron_nvfp4.py     --src /path/to/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4     --dst /path/to/nemotron-3.5-lightning-bf16

Compare the dequant_manifest.json it writes against the one in this repository to confirm you rebuilt the same tensors. The nibble unpacking is bit-exact against compressed-tensors' reference implementation.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "/path/to/nemotron-3.5-lightning-bf16", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "PursuitOfDataScience/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-ToolCall-LoRA")
tok = AutoTokenizer.from_pretrained("/path/to/nemotron-3.5-lightning-bf16")

msgs = [{"role": "user", "content": "What's the weather in Chicago?"}]
tools = [{"type": "function", "function": {
    "name": "get_weather",
    "description": "Current weather for a city.",
    "parameters": {"type": "object",
                   "properties": {"city": {"type": "string"}},
                   "required": ["city"]}}}]
ids = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                              return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=512)[0, ids.shape[1]:]))

The model answers in its native ChatML <tool_call><function=…> syntax. The adapter does not change the interface, only the reliability of using it.

Training

The 23x128 routed experts are deliberately not adapted — they hold 29 of the 30B parameters, and adapting them would defeat the point of a light adapter on a 3B-active model. Validation loss fell 0.5341 → 0.4965 over the run.

Data preparation

Three defects in the raw corpus are filtered, because each teaches the opposite of the goal:

• —7.4% of records leak the corpus generator's own <tool_call>{...}</tool_call> JSON into assistant prose — a third syntax this model's template never uses. Lifted into structured calls.
• —4.2% of gold calls name a tool that was never offered; 114 are literally named unknown. Dropped — supervising those teaches exactly the hallucination this adapter is meant to remove.
• —The reasoning traces interleave the agent's planning with blocks where the data generator invents tool results. Only the agent's own passes are kept; an agent must never be trained to hallucinate observations.

Validation is deduplicated against training by normalised opening prompt. Toucan's subsets overlap by construction, so distinct ids are not distinct problems, and a naive split leaks 4.6% of validation.

Provenance and licensing

• —Base model NVIDIA-Nemotron-3.5-Lightning-30B-A3B, OpenMDW-1.1. That licence requires any redistribution to retain a copy of it and all origin notices.
• —Training data Toucan-1.5M, Apache-2.0.
• —Toucan is synthetic: every trajectory used here was generated by MiniMax-M2.5. The behaviour this adapter installs is distilled from that model's outputs and inherits its conventions.

Limitations

• —Trained on 3,200 examples for 400 steps. Validation loss was still falling, so this is a bounded run, not a converged finetune.
• —The evaluation scores the first tool call of a trajectory. It does not measure multi-turn task completion.
• —Measured only on Toucan-derived data. Gains against a corpus generated by MiniMax-M2.5 are partly gains at matching that model's conventions; this adapter has not been evaluated on an independent tool-calling benchmark.
• —Single seed. Sampling error is roughly ±6.9 points per cell at n=200.
• —The MTP (multi-token-prediction) head survives dequantisation but is not exercised by this recipe; speculative decoding was not re-measured.