TaterTotterson/gemma-4-26B-A4B-it-GGUF-Tater-NoThink

gemma-4-26B-A4B-it-GGUF Tater NoThink

This is a Tater NoThink test build of unsloth/gemma-4-26B-A4B-it-GGUF, bundled with compatible Multi-Token Prediction (MTP) and DFlash draft models for llama.cpp speculative decoding.

The main model weights and projector are from the upstream Unsloth GGUF release. The only intended change to the main text model is its embedded GGUF tokenizer.chat_template metadata:

• —gemma-4-26B-A4B-it-UD-Q4_K_M.gguf

The template is patched to force:

{%- set enable_thinking = false -%}

This makes the model prefer the no-thinking chat-template path even when a client accidentally exposes or passes a thinking flag.

Included Files

• —gemma-4-26B-A4B-it-UD-Q4_K_M.gguf: Tater NoThink patched GGUF.
• —mmproj-F16.gguf: unchanged upstream vision projector.
• —gemma-4-26B-A4B-it-MTP-Q8_0.gguf: Q8_0 MTP draft model converted from Google's official Gemma 4 assistant checkpoint. This is a sidecar draft model, not a standalone chat model.
• —gemma-4-26B-A4B-it-DFlash-Q8_0.gguf: Q8_0 DFlash draft model from the exact Gemma 4 26B-A4B-it DFlash checkpoint. This is also a sidecar draft model, not a standalone chat model.

Speculative Decoding In Tater

Download the main model and one compatible draft model, then configure the llama.cpp provider in Tater.

MTP

1. 1.Select gemma-4-26B-A4B-it-UD-Q4_K_M.gguf as the main model.
2. 2.Enable Speculative Decoding.
3. 3.Set Method to Multi-Token Prediction (MTP).
4. 4.Select gemma-4-26B-A4B-it-MTP-Q8_0.gguf under Draft Model (GGUF).
5. 5.Leave Maximum Draft Tokens at the recommended value of 3, then use Save & Load.

The draft must be paired with this Gemma 4 26B target family. It is not interchangeable with DFlash, DSpark, or drafts made for another target model.

DFlash

1. 1.Select gemma-4-26B-A4B-it-UD-Q4_K_M.gguf as the main model.
2. 2.Enable Speculative Decoding.
3. 3.Set Method to DFlash.
4. 4.Select gemma-4-26B-A4B-it-DFlash-Q8_0.gguf under Draft Model (GGUF).
5. 5.Set Maximum Draft Tokens to 3, then use Save & Load.

The DFlash checkpoint was trained with a block size of 16, but its published mixed-workload benchmark found 3 to be the best general starting value. Structured tasks such as code or JSON may benefit from 4 or 5.

llama.cpp command-line example

Use a recent llama.cpp build with Gemma 4 MTP support:

llama-server \
  --model gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
  --model-draft gemma-4-26B-A4B-it-MTP-Q8_0.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --jinja \
  --flash-attn on

For DFlash, replace the draft model and speculative type:

llama-server \
  --model gemma-4-26B-A4B-it-UD-Q4_K_M.gguf \
  --model-draft gemma-4-26B-A4B-it-DFlash-Q8_0.gguf \
  --spec-type draft-dflash \
  --spec-draft-n-max 3 \
  --jinja \
  --flash-attn on

Actual speedup depends on prompt, hardware, context length, and draft acceptance rate.

Source And Attribution

Upstream repository:

• —https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF
• —https://huggingface.co/google/gemma-4-26B-A4B-it-assistant
• —https://huggingface.co/z-lab/gemma-4-26B-A4B-it-DFlash
• —https://huggingface.co/williamliao/gemma-4-26B-A4B-it-DFlash-GGUF

License:

• —Apache 2.0

This repo is meant as a practical Tater compatibility build, not a new model or fine tune.

Draft model provenance

The MTP sidecar was converted from google/gemma-4-26B-A4B-it-assistant revision 6e5aaaf4c42b98394530b8fda2e95cadd65c151c with llama.cpp revision 8e7f22b67ef4, using Q80 output. During conversion, the source tokenizer configuration's empty `extraspecial_tokens` list was treated as an empty mapping to satisfy the converter; model weights were not changed.

The DFlash sidecar is redistributed unchanged from williamliao/gemma-4-26B-A4B-it-DFlash-GGUF revision d1800ac59f255542ae096018fa696f03918066a6. That conversion is derived from the Apache-2.0-licensed z-lab/gemma-4-26B-A4B-it-DFlash checkpoint and was tested by its publisher with llama.cpp and the same Gemma 4 26B-A4B-it target family.

DSpark Status

A Gemma 4 26B-A4B-it DSpark GGUF exists, but its current SpecForge conversion requires llama.cpp PR #26275 and the dflash.bonus_anchor runtime behavior. That support is not present in the llama.cpp revision bundled with the current Tater release, so the DSpark file is intentionally not mirrored here yet. It will be suitable to add after Tater moves to a compatible llama.cpp revision and passes a paired-model test.

Tater Usage

Use this repo as a llama.cpp GGUF model in Tater. For vision, select mmproj-F16.gguf as the projector.

Recommended first file:

gemma-4-26B-A4B-it-UD-Q4_K_M.gguf