witcheer/gemma-3-27b-eagle3-drafter

gemma-3-27b-eagle3-drafter

an EAGLE-3 speculative-decoding draft head for google/gemma-3-27b-it, trained from scratch on a single RTX 5090 (32GB) over four unattended nights.

pair it with Gemma-3-27B-it in sglang and generation gets faster with no change in output: verified tokens are exactly what the target model would have produced. measured on the training rig, best config (tree-3-4-8), against plain decoding:

(release bench at the selected checkpoint, step 21000; base decode 59.6 tok/s on the same card, code reaching 89.0 tok/s with the drafter by the final bench. zero failed requests across all bench runs.)

usage (sglang)

python -m sglang.launch_server \
  --model-path <your-gemma-3-27b-it path or AWQ variant> \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path witcheer/gemma-3-27b-eagle3-drafter \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 4 \
  --speculative-num-draft-tokens 8

the three configs benched:

verified with sglang 0.5.14, target served as AWQ, context 4096, cuda graphs on.

what's inside

• —716M-parameter single-layer llama-architecture EAGLE-3 head (LlamaForCausalLMEagle3, bf16), hidden size 5376 to match the Gemma-3-27B residual stream
• —draft vocab 32000 with d2t/t2d maps to Gemma's 262k vocabulary
• —no Gemma weights are included; every tensor in this repo was trained from scratch

training

• —framework: SpecForge
• —data: a 54k-sample chat and code dataset
• —hardware: one RTX 5090 (32GB), four nights of otherwise idle time (23:00 to ~05:20 each), ~25h GPU total
• —checkpoint selection: every checkpoint was release-benched live; the speedup curve peaked at step 21000 (39% of one epoch) and regressed at 28000 (prose 1.33x to 1.24x), so 21000 ships. training loss alone does not show this: bench your checkpoints.

curve at a glance (tree-3-4-8):

licence

this head was trained on hidden states produced by Gemma-3-27B-it, so it is distributed as a Gemma model derivative under the Gemma Terms of Use. the drafter never generates final output on its own; all emitted tokens are verified by the Gemma target model.