nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2
Model Overview
Description
NVIDIA-Nemotron-3-Super-120B-A12B-BF16-MTPv2 is an updated, separately distributed Multi-Token Prediction (MTP) head for speculative decoding with NVIDIA Nemotron 3 Super. It is not a standalone language model and does not contain the target model's backbone weights.
The checkpoint was initialized from the built-in MTP head in `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16` and further trained with NVIDIA Megatron-LM, using the same Megatron training infrastructure used for supervised fine-tuning of Nemotron models. The Nemotron 3 Super backbone remained frozen during this additional MTP training.
The MTPv2 head can be used with the following Nemotron 3 Super target checkpoints:
- `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16`
- `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8`
- `nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4`
During speculative decoding, the MTP head proposes draft tokens and the target model verifies them. The target model determines the accepted output, so adopting this checkpoint does not replace or modify the target model.
This model is ready for commercial or non-commercial use.
License/Terms of Use
Governing Download Terms: Use of this model/checkpoint is governed by the NVIDIA Nemotron Open Model License.
Deployment Geography
Global <br>
Use Case
This checkpoint is intended for developers serving NVIDIA Nemotron 3 Super who want to improve inference performance through opt-in speculative decoding. It must be used together with a compatible Nemotron 3 Super target checkpoint and a serving runtime that supports loading a separate MTP head. <br>
Release Date
Hugging Face 08/11/2026 via https://huggingface.co/nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2 <br>
References
- NVIDIA Nemotron 3 Super 120B-A12B BF16 model card
- NVIDIA Nemotron 3 Super Technical Report
- NVIDIA Megatron-LM
- Nemotron Post-Training Dataset v2
- SPEED-Bench
Model Architecture
Architecture Type: Multi-Token Prediction draft head for speculative decoding <br> Network Architecture: Nemotron 3 Super MTP head, trained with Megatron-LM <br> Target Model Architecture: Mamba-2, Mixture-of-Experts, and Attention hybrid with 120B total parameters and 12B active parameters <br> Number of MTP Head Parameters: 3B <br> Initialization and Training: Initialized from the built-in Nemotron 3 Super MTP head and further trained while the target backbone remained frozen. <br>
Input
Input Type(s): Text <br> Input Format(s): String <br> Input Parameters: One Dimensional (1D) <br> Other Properties Related to Input: Maximum context length up to 262144 tokens. Supported languages include: English, French, German, Italian, Japanese, Spanish, and Chinese <br>
Output
Output Type(s): Text <br> Output Format: String <br> Output Parameters: One-Dimensional (1D) <br> Other Properties Related to Output: None <br>
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware and software frameworks, the model can accelerate inference compared with unaccelerated autoregressive decoding when used in a supported configuration. <br>
Software Integration
Supported Runtime Engine(s): vLLM, SGLang, TensorRT-LLM <br>
Supported Hardware Microarchitecture Compatibility: <br>
- NVIDIA Blackwell <br>
- NVIDIA Hopper <br>
Preferred Operating System(s): <br>
- Linux <br>
The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Iterative testing and validation at both model and system levels are essential to mitigate risks and confirm that speculative decoding improves the intended workload.
Model Version(s)
MTPv2 is a separately distributed, updated MTP head for NVIDIA Nemotron 3 Super. It was trained with Megatron-LM. <br>
Training and Evaluation Datasets
Training Dataset
Link: `nvidia/Nemotron-Post-Training-Dataset-v2` <br> Data Modality: Text <br> Text Training Data Size: Refer to the source dataset card <br> Data Collection Method by Dataset: Hybrid: Automated, Synthetic <br> Labeling Method by Dataset: Hybrid: Automated, Synthetic <br> Properties: The MTPv2 head was initialized from the built-in Nemotron 3 Super MTP head and further trained using Nemotron Post-Training Dataset v2. The source dataset combines prompts sourced from public and open corpora or synthetically generated prompts; prompts are filtered for quality and complexity, and responses are synthetically generated by public and open models. <br>
Evaluation Dataset
Link: `nvidia/SPEED-Bench` <br> Data Collection Method by Dataset: Hybrid: Manually-Labeled, Synthetic <br> Labeling Method by Dataset: Hybrid: Manually-Labeled, Synthetic <br> Properties: SPEED-Bench contains 880 curated multi-turn dialogue sequences across 11 categories, with 80 samples per category. Mean acceptance length was measured on the qualitative split with draft length 7 and temperature 0. <br>
Paired target-model quality sanity checks on GPQA, HLE, MMLU-Pro, and AIME 2025 were closely aligned between the built-in MTP head and MTPv2. Because exact speculative decoding verifies draft tokens with the target model, the target checkpoint remains responsible for the final output distribution.
Inference
Acceleration Engine: vLLM <br> Test Hardware: NVIDIA Hopper and Blackwell GPUs <br>
Multi-Token Prediction Speculative Decoding
The MTPv2 head predicts multiple candidate tokens beyond the next token. During generation, the target Nemotron 3 Super model verifies these candidates and accepts the longest valid prefix. The number of accepted tokens per target-model verification step is called the acceptance length. Higher acceptance length can reduce the number of target-model decoding steps, but end-to-end speedup depends on hardware, workload, concurrency, context length, serving configuration, and draft length.
On the SPEED-Bench qualitative split with draft length 7 and temperature 0, the updated MTPv2 head improved mean acceptance length from approximately 3.45 to 4.31 accepted tokens per speculative step relative to the built-in MTP head.
Usage
The following example serves the BF16 target with the separate MTPv2 head in vLLM. Tested on v20.22.0 vLLM version.
vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 \
--tensor-parallel-size 8 \
--trust-remote-code \
--speculative-config '{
"method": "mtp",
"model": "nvidia/Nemotron-3-Super-120B-A12B-BF16-MTPv2",
"num_speculative_tokens": 7
}'The target model's normal serving arguments, chat template, reasoning parser, tool-call parser, sampling parameters, and context-length settings continue to apply. Users should benchmark the draft length and serving configuration on their own workload.
Model Limitations
- This checkpoint is not a standalone language model and cannot generate user-visible output without a compatible Nemotron 3 Super target checkpoint.
- Compatibility depends on the serving runtime correctly supporting a separately distributed MTP head.
- The checkpoint inherits the capabilities, limitations and safety considerations of the target Nemotron 3 Super model.
- Serving or kernel defects can affect correctness or performance even though exact speculative decoding is designed to preserve the target model's output distribution.
- The MTP head does not provide independent safety controls. Deployers must retain target-model safeguards, application guardrails, least-privilege access, and runtime security validation.
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.
Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.
SUBCARDS: