soyrsoyr/Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-NVFP4A16-pr3118-validation

Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-NVFP4A16-pr3118-validation

H100 load and generation PASSED with actual MTP draft-token metrics. This is not a quality or performance benchmark.

Dense-source MTP was quantized using the requested data-free scheme.

Provenance and format

• —Source: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.
• —Implementation: llm-compressor PR 3118, `87347881`.
• —Backbone and MTP formats are separate; inspect config.json, recipe.yaml when present, and pr3118-validation.json. NVFP4A16 is weight-only FP4 with 16-bit activations, not calibrated NVFP4 W4A4. MXFP4 uses dynamic activation quantization.
• —Source licensing remains applicable; consult the upstream model card. This validation adds no license grant.

End-to-end MTP test

Validated runtime baseline: vllm==0.29.1rc1.dev79+g767d1c4d4, Transformers 5.17.0, CUDA 13.0. MXFP4 requires your B200 run to establish runtime compatibility.

vllm serve soyrsoyr/Nemotron3.5-Lightning-30B-A3B-FP8Dyn-MTP-NVFP4A16-pr3118-validation --dtype bfloat16 --max-model-len 1024 --enforce-eager --gpu-memory-utilization 0.85 --speculative-config '{"method":"mtp","num_speculative_tokens":1}' --limit-mm-per-prompt '{"image":0,"video":0}'

After downloading this snapshot, python verify_mtp.py /path/to/snapshot runs two prompts and requires positive draft-token metrics. A successful model load alone is not counted as an MTP pass. The script records generations and speculative decoding metrics.