QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF

Gemma 4 E4B — 2× lower KL than Google's official Q4_0 QAT

   

Native QUASAR Q40 for llama.cpp / Ollama / LM Studio: **0.044 KL to BF16 vs 0.088** for [Google's official Q40 QAT](https://huggingface.co/google/gemma-4-E4B-it-qat-q40-gguf/blob/main/gemma-4-E4Bq40-it.gguf), with standard Q40 runtime.

QUASAR Q40 also has lower KL than [Unsloth's Q40 repack of Google's QAT](https://huggingface.co/unsloth/gemma-4-E4B-it-qat-GGUF/blob/main/gemma-4-E4B-it-qat-UD-Q4KXL.gguf) (0.071; the UD-Q4_K_XL file is Q40 tensors despite its name) and [LM Studio's Q4KM](https://huggingface.co/lmstudio-community/gemma-4-E4B-it-GGUF/blob/main/gemma-4-E4B-it-Q4K_M.gguf) (0.114), a more expressive format. Same BF16 reference, prompts, llama.cpp build and backend for every row.

The W4A16 twin shows the same pattern under vLLM: KL 0.022 to BF16 vs 0.064 for Google's W4A16 QAT.

ollama run hf.co/QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF
llama-cli -hf QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF

[image]

Downstream benchmarks

Higher point estimates on all 8 downstream benchmarks than Google's QAT model (measured on the W4A16 twin, which carries the same trained codes).

Same evaluation harness; full configs, BF16 results and paired-bootstrap CIs are in `EVAL.md`.

Native QAT GGUF

Native QAT checkpoint: the GGUF stores QUASAR's trained INT4 codes and scales directly, with no post-training re-quantization. All 258 quantized tensors are verified bit-exact against the W4A16 twin; see `native_q4_0_receipt.json` and the included verifier (scripts/export_gemma4_gguf_q4_0.py --verify).

Standard Q4_0 runtime in llama.cpp, Ollama and LM Studio. Vision is included through the BF16 projector.

Run it

• —Ollama: ollama run hf.co/QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF — the vision projector is picked up automatically.
• —LM Studio: search QUASAR-QAT gemma-4-E4B in Discover.
• —llama.cpp, vision: llama-mtmd-cli -hf QUASAR-QAT/gemma-4-E4B-it-QUASAR-Q4_0-GGUF --image photo.jpg -p "Describe this image." --jinja
• —Thinking: on by default in the embedded Gemma 4 chat template; toggle with --chat-template-kwargs '{"enable_thinking": false}' (llama-cli / llama-server).
• —Context: 131,072 tokens.
• —Files: gemma-4-E4B-it-QUASAR-Q4_0.gguf (5.23 GB — Q40 linears with trained group-64 scales, Q6K token embeddings, chat template embedded) and mmproj-gemma-4-E4B-it-BF16.gguf (992 MB, the unmodified BF16 vision projector from `google/gemma-4-E4B-it`).

QUASAR

QUASAR is loss-aware quantization-aware training. It improves the low-bit reconstruction during training, then exports to standard deployment formats with zero inference overhead.

This checkpoint was healed against the BF16 teacher for 626M tokens: one epoch over ~398k prompts self-distilled from BF16 Gemma 4 E4B-it with thinking on, fp32 master weights, INT4 group-64.

Related

• —W4A16 compressed-tensors (vLLM) twin of this checkpoint: QUASAR-QAT/gemma-4-E4B-it-QUASAR-W4A16-G64
• —Gemma 4 12B, same recipe: GGUF · W4A16 · collection · Gemma 4 E4B & 12B — QUASAR 4-bit QAT
• —Base model: google/gemma-4-E4B-it · Paper: arXiv:2608.13966 · Org: huggingface.co/QUASAR-QAT

License

Based on Gemma 4 by Google DeepMind. Upstream LICENSE/NOTICE apply.

Citation

arxiv.org/abs/2608.13966

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}