Modelpublic

glux-cz/Qwen3-8B-NVFP4-Blackwell

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face

1likes43downloads

Model Card

Qwen3-8B-NVFP4

NVFP4 quantized Qwen3-8B for NVIDIA Blackwell GPUs (RTX 5090, RTX PRO 4000).

Details

—Format: NVFP4 (4-bit FP) + FP8 KV cache
—Tools: TensorRT-LLM 1.2.0, ModelOpt 0.37.0
—Calibration: 512 samples, cnn_dailymail

Usage

bash

huggingface-cli download glux-cz/Qwen3-8B-NVFP4-Blackwell --local-dir ./checkpoint
trtllm-build --checkpoint_dir ./checkpoint --output_dir ./engine --gemm_plugin nvfp4