Modelpublic

migtissera/Tess-4-27B-EAGLE3

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

12likes52downloads

Model Card

Tess-4-27B · EAGLE-3 Draft

An EAGLE-3 speculative-decoding draft head for [migtissera/Tess-4-27B](https://huggingface.co/migtissera/Tess-4-27B) — trained on-policy against Tess-4's own outputs, so it predicts this model, not its base.

Measured: 1.76× faster decoding on average — up to 2.4× on reasoning. Lossless: the target model verifies every token, so outputs are identical to non-speculative decoding.

Benchmarks

Single H100-80GB, greedy decoding, 10 held-out prompts × 1024 tokens each:

Workload	Baseline	With EAGLE-3	Speedup
Math / logic reasoning	51.2 tok/s	103–123 tok/s	2.0–2.4×
Coding	51.2 tok/s	81–91 tok/s	1.6–1.8×
Agentic planning	51.2 tok/s	83 tok/s	1.6×
General chat	51.2 tok/s	80–87 tok/s	1.6–1.7×
Overall	51.2 tok/s	90.2 tok/s	1.76×

Server-reported accept length: 1.5–2.2 tokens per verification step (conservative settings: topk=1, 4 draft tokens — tuning headroom remains).

Why this exists

Tess-4-27B inherits an MTP head from its base model, but fine-tuning shifts the output distribution, driving that head's acceptance to ~0% — it can't accelerate the fine-tune. This EAGLE-3 draft was trained against Tess-4 itself: ~4,700 on-policy samples (Tess-4's own reasoning + coding + agentic generations, including real multi-turn tool-call traces), 2 epochs, with the draft consuming fused hidden states from three depths of the frozen target.

Usage

SGLang (tested ✅)

bash

python -m sglang.launch_server \
  --model-path migtissera/Tess-4-27B \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path migtissera/Tess-4-27B-EAGLE3 \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --trust-remote-code

vLLM

bash

vllm serve migtissera/Tess-4-27B \
  --speculative-config '{"method": "eagle3", "model": "migtissera/Tess-4-27B-EAGLE3", "num_speculative_tokens": 4}' \
  --trust-remote-code

(Benchmarked with SGLang; vLLM flags may vary by version.)

Details

—Architecture: 1-layer Llama-style EAGLE-3 head (hidden 5120, matching the target), draft vocab 32K mapped onto the target's 248K vocab. 1.22 GB. Input embeddings are not included — the serving engine shares them from the target model.
—Training: SpecForge (SGLang team), online mode, frozen target, single H100.
—Data: on-policy — the target generated every training token. Mix: agentic/coding prompts from the Tess-4 program, OpenCoder, GSM8K, UltraChat, plus real multi-turn tool-call traces.
—Lossless: speculative decoding never changes outputs; the target verifies every proposed token. Speedup depends on workload (structured/reasoning content accepts more).

License

Apache 2.0, matching Tess-4-27B.

Part of the Tess series by [Migel Tissera](https://huggingface.co/migtissera).