Modelpublic

shiny-plan/Nemotron-Cascade-2-30B-A3B-Q4_K_M-GGUF

sourceHugging Faceotherupdated 6mo agoView on Hugging Face

1likes34downloads

Model Card

Nemotron-Cascade-2-30B-A3B — Q4KM GGUF

GGUF quantization of nvidia/Nemotron-Cascade-2-30B-A3B.

—Architecture: Hybrid Attention + Mamba (SSM) + MoE — 30B total parameters, 3B active
—Quantization: Q4KM (k-quant, mixed precision ~4.5 bpw)

Quantization commands

bash

# Convert HF model to GGUF (bf16)
python llama.cpp/convert_hf_to_gguf.py \
  nvidia/Nemotron-Cascade-2-30B-A3B \
  --outfile Nemotron-Cascade-2-30B-A3B-bf16.gguf \
  --outtype bf16

# Quantize to Q4_K_M
llama-quantize Nemotron-Cascade-2-30B-A3B-bf16.gguf \
  Nemotron-Cascade-2-30B-A3B-Q4_K_M.gguf Q4_K_M

Usage

Load in LM Studio, llama.cpp, or any GGUF-compatible runtime.