inference-optimization/Qwen3-8B-speculators.peagle-qwen3arch-ckpt4

Qwen3-8B P-EAGLE Drafter (Qwen3 Architecture)

P-EAGLE (Parallel Eagle) speculative decoding drafter for Qwen/Qwen3-8B using the Qwen3 draft architecture (--draft-arch qwen3).

The Qwen3 architecture uses additional q_norm/k_norm layers in attention, which stabilize training at higher learning rates compared to the default Llama draft architecture. See speculators#563 for the RFC and experimental results.

Benchmark Results

Evaluated on RedHatAI/speculator_benchmarks (9 subsets) via vLLM throughput mode.

Acceptance lengths

Comparison vs Llama-arch baseline

Format: delta vs Llama-arch P-EAGLE.

Qwen3-arch outperforms Llama-arch on all 9 subsets at k=5 and 8/9 at k=7.

Training

Usage

Requires vLLM with Qwen3 Eagle3/P-EAGLE support (vllm#43132) and the architecture resolution fix.

vllm serve Qwen/Qwen3-8B \
  --speculative-config '{
    "model": "inference-optimization/Qwen3-8B-speculators.peagle-qwen3arch-ckpt4",
    "num_speculative_tokens": 7,
    "method": "eagle3",
    "parallel_drafting": true
  }'

Related

• —speculators#563 — RFC: Support Qwen3 base architecture for Eagle3 & P-EAGLE
• —Llama-arch P-EAGLE baseline — Same setup with Llama draft architecture
• —Eagle3 Qwen3-arch drafter — Eagle3 (sequential) variant