gsrunion/Nemotron-3-Nano-30B-A3B-ROCmFP4-STRIX_LEAN-GGUF

Nemotron-3 Nano 30B-A3B — ROCmFP4 STRIX_LEAN for AMD Strix Halo

ROCmFP4 GGUF quantization of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B (hybrid Mamba-MoE, nemotron_h_moe, 31.6B params / ~3.5B active), produced for AMD Strix Halo (Ryzen AI Max, gfx1151).

To our knowledge this is the first ROCmFP4-family quant of a hybrid Mamba-MoE architecture — the recipe carries over cleanly (1 of 401 tensors required fallback quantization).

⚠️ Stock llama.cpp will reject this file (invalid ggml type 106). It runs on: - the kyuz0/amd-strix-halo-toolboxes rocm-7.2.4-rocmfp4 container image (used to make and bench it), or - the Hal0ai/Hal0_ROCmFPX llama.cpp fork.

Companion repos: Ornith-1.0-35B · Ornith-1.0-9B · Qwen3.6-35B-A3B

Files

Measured performance

On AMD Ryzen AI Max+ 395 (Strix Halo, 128 GB unified LPDDR5X, ROCm backend, llama-bench -fa 1 --mmap 0, same box and build for both rows):

+27% decode at 31% smaller. Decode on Strix Halo is bandwidth-bound; the FP4 kernels stream fewer bytes per token and sustain near-peak effective bandwidth on gfx1151.

How it was made

BF16 GGUF source and imatrix from unsloth/Nemotron-3-Nano-30B-A3B-GGUF, quantized with the toolbox image's llama-quantize (build 9219):

llama-quantize --imatrix imatrix_unsloth.gguf_file \
  Nemotron-3-Nano-30B-A3B-BF16-00001-of-00002.gguf \
  Nemotron-3-Nano-30B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN

Serving

llama-server -m Nemotron-3-Nano-30B-A3B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  -ngl 999 -fa on --jinja -c 65536

Credits

• —Base model: NVIDIA — Nemotron-3 Nano 30B-A3B (NVIDIA Open Model License)
• —BF16 GGUF conversion + imatrix: unsloth
• —ROCmFP4 quant formats: Hal0ai; GPU quantizer build: kyuz0