QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF

Gemma 4 12B — closest Q4_0 to BF16

   

30% lower KL than [Google's official Q4_0 QAT](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf/blob/main/gemma-4-12b-it-qat-q4_0.gguf) and 2.2× closer to BF16 than [Unsloth's imatrix Q4_0](https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/blob/main/gemma-4-12b-it-Q4_0.gguf). Same size as Google. Standard Q4_0 runtime.

The W4A16 twin shows the same pattern under transformers: KL 0.034 to BF16 vs 0.044 for Google's W4A16 QAT (top-1 95.5% vs 94.8%).

ollama run hf.co/QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF
llama-cli -hf QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF

[image]

Better fidelity than the popular alternatives

QUASAR also gets lower KL than Unsloth's Q4_0 repack of Google's QAT checkpoint, and than LM Studio's Q4_K_M — a more expressive format:

Same BF16 reference, prompts, llama.cpp build, and backend.

Full evaluation and reproduction details → EVAL.md

Downstream benchmarks

Higher point estimates on all four downstream benchmarks than Google's QAT model (measured on the W4A16 twin, which carries the same trained codes).

The MMLU gain is statistically robust: +2.6 points, 95% CI [+2.0, +3.2] (paired bootstrap over 14,042 questions). Full configs, BF16 results and CIs are in `EVAL.md`.

Native QAT GGUF

Native Q40 QAT, not a post-training re-quantization. The GGUF stores QUASAR's trained INT4 codes and scales directly; all 328 quantized tensors are verified against the [W4A16 twin](https://huggingface.co/QUASAR-QAT/gemma-4-12B-it-QUASAR-W4A16-G64) ([`nativeq40receipt.json](https://huggingface.co/QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF/blob/main/native_q4_0_receipt.json), verifier in scripts/`). No re-rounding and no custom runtime.

Multimodal support is included through the BF16 vision projector from google/gemma-4-12B-it.

Run it

• —Ollama: ollama run hf.co/QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF — the vision projector is picked up automatically.
• —LM Studio: search QUASAR-QAT gemma-4-12B in Discover.
• —llama.cpp, vision: llama-mtmd-cli -hf QUASAR-QAT/gemma-4-12B-it-QUASAR-Q4_0-GGUF --image photo.jpg -p "Describe this image." --jinja
• —Thinking: on by default in the embedded Gemma 4 chat template; toggle with --chat-template-kwargs '{"enable_thinking": false}' (llama-cli / llama-server).
• —Sampling: Gemma 4 defaults — temperature 1.0, top-k 64, top-p 0.95.
• —Context: 262,144 tokens.

What's inside

QUASAR

QUASAR is loss-aware quantization-aware training. It improves the low-bit reconstruction during training, then exports to standard deployment formats with zero inference overhead.

This checkpoint was healed against the BF16 teacher for ~600M tokens: one epoch over ~377k prompts self-distilled from BF16 Gemma 4 12B-it with thinking on, fp32 master weights, INT4 group-64.

Related

• —W4A16 compressed-tensors (vLLM) twin of this checkpoint: QUASAR-QAT/gemma-4-12B-it-QUASAR-W4A16-G64
• —Gemma 4 E4B, same recipe: GGUF · W4A16 · collection · Gemma 4 E4B & 12B — QUASAR 4-bit QAT
• —Base model: google/gemma-4-12B-it · Paper: arXiv:2608.13966 · Org: huggingface.co/QUASAR-QAT

License

Based on Gemma 4 by Google DeepMind. Upstream LICENSE/NOTICE apply.

Citation

arxiv.org/abs/2608.13966

@article{counathe2026quasar,
  title={QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction},
  author={Counathe, Vincent and Athiwaratkun, Ben and De Sa, Christopher and Zhang, Tianyi},
  journal={arXiv preprint arXiv:2608.13966},
  year={2026}
}