inference-optimization/Qwen3-8B-from-Qwen3-8B_regen-speculators.eagle3-qwen3arch-ckpt1

Qwen3-8B Eagle3 Drafter (Qwen3 Architecture)

Eagle3 speculative decoding drafter for Qwen/Qwen3-8B using the Qwen3 draft architecture (--draft-arch qwen3).

The Qwen3 architecture uses additional q_norm/k_norm layers in attention, which stabilize training at higher learning rates compared to the default Llama draft architecture. See speculators#563 for the RFC and experimental results.

Benchmark Results

Evaluated with sampling params temp=0.6, topp=0.95, topk=20.

Acceptance lengths

Comparison vs Llama-arch baseline

Trained on the same dataset with the same hyperparameters (except LR). Format: delta vs llamaarch-ckpt1.

Outperforms the Llama-arch baseline by ~3-7% across all benchmarks and all k values, with the gap widening at higher k (math k=7: 3.68 vs 3.45, +6.7%).

Training

Usage

Requires vLLM with Qwen3 Eagle3 support (vllm#43132) and the architecture resolution fix.

vllm serve Qwen/Qwen3-8B \
  --speculative-config '{
    "model": "inference-optimization/Qwen3-8B-from-Qwen3-8B_regen-speculators.eagle3-qwen3arch-ckpt1",
    "num_speculative_tokens": 3,
    "method": "eagle3"
  }'

Related

• —speculators#563 — RFC: Support Qwen3 base architecture for Eagle3 & P-EAGLE
• —Llama-arch baseline — Same setup with Llama draft architecture
• —LR sweep results — Llama vs Qwen3 draft arch comparison across 5 learning rates