majentik/Nemotron-Cascade-2-30B-A3B-TurboQuant-GGUF-IQ4_XS

[!TIP] KV-cache quantization (upstream, no fork needed): llama.cpp/Ollama cover this natively — -ctk q8_0 -ctv q8_0 (~half KV memory, negligible quality loss) or -ctk q4_0 -ctv q4_0 (~quarter memory, small quality cost). In Ollama: OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1.

Nemotron-Cascade-2-30B-A3B — TurboQuant GGUF IQ4_XS

`nvidia/Nemotron-Cascade-2-30B-A3B` quantized pack, published as Nemotron-Cascade-2-30B-A3B-TurboQuant-GGUF-IQ4_XS.

Method

llama.cpp IQ4_XS quantization.

Release line

Released under the TurboQuant line. RotorQuant and TurboQuant are this project's release labels for this pack, not distinct quantization algorithms — both brand repos carry byte-identical weights. No brand-specific speedup is claimed or measured.

Modality

pipeline_tag: text-generation. This is a Mixture-of-Experts (MoE) model — a subset of experts is active per token; total and active parameter counts differ. This is a llama.cpp GGUF conversion of the text tower; no modality beyond pipeline_tag above is claimed or included.

License

This pack is a derivative of `nvidia/Nemotron-Cascade-2-30B-A3B`; all credit for the original model, training, and weights belongs to the upstream authors. This repo republishes a quantized conversion of those weights only.

Governed by the nvidia-open-model-license. See the upstream repo and the linked license for the full terms — no license text is reproduced here.