Modelpublic

dynamicagency/gemma-4-26b-a4b-it-oQ8e-mtp-drafter

sourceHugging Faceupdated 2mo agoView on Hugging Face

1likes658downloads

Model Card

gemma-4-26b-a4b-it-oQ8e-mtp-drafter

MLX conversion of Google's official MTP ("assistant") drafter checkpoint for gemma-4-26B-A4B-it, converted from google/gemma-4-26B-A4B-it-assistant. Kept at bf16, matching Google's original release precision — drafters are small enough that quantizing them further trades draft-acceptance accuracy for little practical size/speed benefit.

Usage

Pair with the main model, `dynamicagency/gemma-4-26b-a4b-it-oQ8e-mtp`. This is a separate loadable model, not a set of tensors merged into the main checkpoint — Gemma 4's MTP mechanism loads a small assistant model alongside the main one at inference time rather than baking draft-head tensors into the same checkpoint (the approach some other model families use).

In oMLX, discover this model directory, then on the main model's settings:

json

{
  "vlm_mtp_enabled": true,
  "vlm_mtp_draft_model": "<local id of this drafter model>"
}

—Model type: gemma4_assistant
—Format: MLX safetensors, bfloat16