QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF

Muse Glimmer 30B — native Q4_0 QAT for llama.cpp

   

Native Q4_0 quantization-aware training with QUASAR. On held-out response tokens, this checkpoint is closer to BF16 than Meta's official Q4_K_M GGUF: 0.0213 vs 0.0228 KL and 95.9% vs 95.6% top-1 agreement, while training all 416 decoder projections directly on the standard Q4_0 lattice.

[NVFP4 / vLLM](https://huggingface.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4) · [W4A4 / Blackwell](https://huggingface.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-NVFP4-W4A4) · GGUF / llama.cpp (this repo) · [Muse Glimmer collection](https://huggingface.co/collections/QUASAR-QAT/quasar-native-4-bit-muse-glimmer-6aa4736ca711a87d471e807d) · [QUASAR paper](https://arxiv.org/abs/2608.13966)

Measured on 948 held-out agentic prompts and 1.19M response tokens with the same fidelity harness.

Run it

ollama run hf.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF

llama-server -hf QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q4_0-GGUF \
  -c 131072 -fa on --jinja

For LM Studio, search for QUASAR-QAT Muse-Glimmer.

Supports 131,072-token context with the embedded Muse chat template for reasoning and tool calls. Requires a llama.cpp build with Muse Glimmer support (9a4843c, September 2026, or newer). This GGUF is text-only.

Selected benchmarks

Both models were served through the same llama.cpp build with identical prompts and decoding. Full same-harness fidelity and downstream results are available in `eval/`.

Technical details

All 416 decoder projections were trained directly on the Q4_0 lattice with QUASAR: INT4 codes with one scale per 32-weight block. The GGUF packs the trained codes and scales directly rather than applying a separate post-training quantization pass.

The checkpoint is 19.6 GB, with Q40 decoder projections, F16 token embeddings and output head, F32 norms, and the Muse chat template embedded. Export fidelity is documented in [`nativeq40receipt.json`](https://huggingface.co/QUASAR-QAT/Muse-Glimmer-30B-QUASAR-Q40-GGUF/blob/main/nativeq40receipt.json).

QUASAR is loss-aware quantization-aware training: it improves low-bit reconstruction during training and exports to standard deployment formats with zero inference overhead.

Related

NVFP4 / vLLM · NVFP4 W4A4 / Blackwell · Muse Glimmer 30B — QUASAR 4-bit QAT · Base model · QUASAR paper

Running this checkpoint? Share benchmark results, deployment notes, or integrations in Discussions.

Citation

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}