soyrsoyr/Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-MXFP4-FromFP8MTP-pr3118-validation

Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-MXFP4-FromFP8MTP-pr3118-validation

B200 runtime validation PENDING. Conversion and checkpoint consistency checks passed; no MXFP4 inference pass is claimed.

Native FP8 MTP was dequantized and quantized to the requested FP4 format.

The native-FP8 input was a derived test checkpoint: BF16 backbone retained, MTP converted to native block FP8. It is not an official FP8 release.

Provenance and format

• —Source: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.
• —Implementation: llm-compressor PR 3118, `87347881`.
• —Backbone and MTP formats are separate; inspect config.json, recipe.yaml when present, and pr3118-validation.json. NVFP4A16 is weight-only FP4 with 16-bit activations, not calibrated NVFP4 W4A4. MXFP4 uses dynamic activation quantization.
• —Source licensing remains applicable; consult the upstream model card. This validation adds no license grant.

End-to-end MTP test

Validated runtime baseline: vllm==0.29.1rc1.dev79+g767d1c4d4, Transformers 5.17.0, CUDA 13.0. MXFP4 requires your B200 run to establish runtime compatibility.

vllm serve soyrsoyr/Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-MXFP4-FromFP8MTP-pr3118-validation --dtype bfloat16 --max-model-len 1024 --enforce-eager --gpu-memory-utilization 0.85 --speculative-config '{"method":"mtp","num_speculative_tokens":1}' --limit-mm-per-prompt '{"image":0,"video":0}'

After downloading this snapshot, python verify_mtp.py /path/to/snapshot runs two prompts and requires positive draft-token metrics. A successful model load alone is not counted as an MTP pass. The script records generations and speculative decoding metrics.