RedHatAI/Qwen3-30B-A3B-Thinking-2507-speculator.eagle3

Qwen3-30B-A3B-Thinking-2507-speculator.eagle3

Model Overview

• —Verifier: Qwen3-30B-A3B
• —Speculative Decoding Algorithm: EAGLE-3
• —Model Architecture: Eagle3Speculator
• —Release Date: 3/12/2026
• —Version: 1.0
• —Model Developers: RedHat

This model is a copy of RedHatAI/Qwen3-30B-A3B-speculator.eagle3. It can be used with Qwen/Qwen3-30B-A3B-Thinking-2507 as well.

This model is based on the EAGLE-3 speculative decoding algorithm. It was trained using the speculators library on a combination of the Magpie-Align/Magpie-Pro-300K-Filtered and the HuggingFaceH4/ultrachat_200k datasets. The model was trained with thinking enabled. This model should be used with the Qwen3-30B-A3B chat template, specifically through the /chat/completions endpoint.

Use with vLLM

vllm serve Qwen3-30B-A3B-Thinking-2507 \
  -tp 1 \
  --speculative-config '{
    "model": "RedHatAI/Qwen3-30B-A3B-Thinking-2507-speculator.eagle3",
    "num_speculative_tokens": 5,
    "method": "eagle3"
  }'

Evaluations

Model / run: Qwen3-30B-A3B-Thinking-2507-speculator.eagle3 (CKPT 5) vLLM: 0.15.0 Training data: Magpie + UltraChat; responses from the Qwen/Qwen3-235B-A22B model (with reasoning enabled).

<h3>Acceptance lengths (draft length)</h3> <table> <thead> <tr> <th>Dataset</th> <th>k=1</th> <th>k=2</th> <th>k=3</th> <th>k=4</th> <th>k=5</th> </tr> </thead> <tbody> <tr> <td>HumanEval</td> <td>1.81</td> <td>2.44</td> <td>2.90</td> <td>3.21</td> <td>3.44</td> </tr> <tr> <td>math_reasoning</td> <td>1.84</td> <td>2.50</td> <td>3.02</td> <td>3.41</td> <td>3.70</td> </tr> <tr> <td>qa</td> <td>1.69</td> <td>2.15</td> <td>2.44</td> <td>2.61</td> <td>2.72</td> </tr> <tr> <td>question</td> <td>1.76</td> <td>2.32</td> <td>2.71</td> <td>2.93</td> <td>3.09</td> </tr> <tr> <td>rag</td> <td>1.74</td> <td>2.25</td> <td>2.60</td> <td>2.82</td> <td>2.97</td> </tr> <tr> <td>summarization</td> <td>1.66</td> <td>2.05</td> <td>2.30</td> <td>2.43</td> <td>2.51</td> </tr> <tr> <td>translation</td> <td>1.72</td> <td>2.21</td> <td>2.53</td> <td>2.74</td> <td>2.87</td> </tr> </tbody> </table>

<details> <summary>Details</summary>

<strong>Configuration</strong>

• —Model: Qwen3-30B-A3B-Thinking-2507
• —Data: Magpie + UltraChat — responses from Qwen3-30B-A3B model (reasoning)
• —temperature: 0.0
• —vllm: 0.15.0
• —backend: vLLM chat_completions
• —rate-type: throughput
• —max-seconds per run: 300
• —hardware: 8× GPU (tensor parallel 8)
• —Benchmark data: RedHatAI/speculator_benchmarks
• —vLLM serve: --no-enable-prefix-caching, --max-num-seqs 64, --enforce-eager

<strong>Command</strong>

GUIDELLM__PREFERRED_ROUTE="chat_completions" \
GUIDELLM__MAX_CONCURRENCY=128 \
guidellm benchmark \
  --target "http://localhost:8000/v1" \
  --data "RedHatAI/speculator_benchmarks" \
  --data-args '{"data_files": "HumanEval.jsonl"}' \
  --rate-type throughput \
  --max-seconds 300

GuideLLM interface changed, so for compatibility with the latest version (v0.6.0), please use the following command:

GUIDELLM__PREFERRED_ROUTE="chat_completions" \
guidellm benchmark \
  --target "http://localhost:8000/v1" \
  --data "RedHatAI/speculator_benchmarks" \
  --data-args '{"data_files": "HumanEval.jsonl"}' \
  --profile sweep \
  --max-seconds 1800 \
  --output-path "my_output.json" \
  --backend-args '{"extras": {"body": {"temperature":0.6, "top_p":0.95, "top_k":20}}}'

</details>