Modelpublic

RedHatAI/gemma-4-31B-it-speculator.eagle3

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

52likes1.8kdownloads

Model Card

gemma4-31B-it-speculator.eagle3

This is a preliminary model release, we will continue to train the model and improve the acceptance rates in the next few days.

Model Overview

—Verifier: google/gemma-4-31b-it
—Speculative Decoding Algorithm: EAGLE-3
—Model Architecture: Eagle3Speculator
—Release Date: 04/09/2026
—Version: 1.0
—Model Developers: RedHat

This is a speculator model designed for use with google/gemma-4-31b-it, based on the EAGLE-3 speculative decoding algorithm. It was trained using the Speculators library on a combination of the Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered dataset and the train_sft split of the HuggingFaceH4/ultrachat_200k dataset. Training data used Magpie + UltraChat with responses from the gemma-4-31B-it model (no reasoning). This model should be used with the google/gemma-4-31b-it chat template, specifically through the /chat/completions endpoint.

vLLM version

UPDATE: Now supported on vllm-main!

Use with vLLM

bash

vllm serve google/gemma-4-31b-it \
  --tensor-parallel-size 2 \
  --speculative-config '{
    "model": "RedHatAI/gemma-4-31B-it-speculator.eagle3",
    "num_speculative_tokens": 3,
    "method": "eagle3"
  }' \
  --max-num-seqs 64 \

Evaluations

Model / run: vLLM: UPDATE: Now supported on vllm-main!

Training data: Magpie + UltraChat; responses from the gemma 4 31B it model (no reasoning).

<h3>Use cases</h3> <table> <thead> <tr> <th>Use Case</th> <th>Dataset</th> <th>Number of Samples</th> </tr> </thead> <tbody> <tr> <td>Coding</td> <td>HumanEval</td> <td>164</td> </tr> <tr> <td>Math Reasoning</td> <td>mathreasoning</td> <td>80</td> </tr> <tr> <td>Question Answering</td> <td>qa</td> <td>80</td> </tr> <tr> <td>MTbench (Question)</td> <td>question</td> <td>80</td> </tr> <tr> <td>RAG</td> <td>rag</td> <td>80</td> </tr> <tr> <td>Summarization</td> <td>summarization</td> <td>80</td> </tr> <tr> <td>Translation</td> <td>translation</td> <td>80</td> </tr> </tbody> </table>

<h3>Acceptance lengths (draft length, temperature=default)</h3> <table> <thead> <tr> <th>Dataset</th> <th>k=1</th> <th>k=2</th> <th>k=3</th> <th>k=4</th> <th>k=5</th> </tr> </thead> <tbody> <tr> <td>HumanEval</td> <td>1.86</td> <td>2.55</td> <td>3.10</td> <td>3.50</td> <td>3.80</td> </tr> <tr> <td>math_reasoning</td> <td>1.87</td> <td>2.59</td> <td>3.15</td> <td>3.59</td> <td>3.93</td> </tr> <tr> <td>qa</td> <td>1.64</td> <td>2.01</td> <td>2.22</td> <td>2.34</td> <td>2.38</td> </tr> <tr> <td>question</td> <td>1.73</td> <td>2.21</td> <td>2.53</td> <td>2.71</td> <td>2.83</td> </tr> <tr> <td>rag</td> <td>1.72</td> <td>2.21</td> <td>2.50</td> <td>2.65</td> <td>2.80</td> </tr> <tr> <td>summarization</td> <td>1.60</td> <td>1.92</td> <td>2.07</td> <td>2.15</td> <td>2.20</td> </tr> <tr> <td>translation</td> <td>1.69</td> <td>2.13</td> <td>2.41</td> <td>2.57</td> <td>2.68</td> </tr> </tbody> </table>