Modelpublic

DaoCloud/Qwen3.8-27B-DFlash2-Exp

sourceHugging Faceapache-2.0updated 23d agoView on Hugging Face

1likes439downloads

Model Card

Qwen3.8-27B-DFlash2-Exp

An experimental DFlash2 drafter for `Qwen/Qwen3.8-27B`.

This checkpoint uses a 7-query sample_from_anchor=true layout that produces seven draft proposals from seven query positions instead of eight, reducing draft-side query compute.

Results

Setting	Value
Engine	vLLM
Hardware	NVIDIA H200, TP1
Concurrency	1
Thinking sampling	`temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=0.0`, `repetition_penalty=1.0`
Thinking mode	`reasoning_effort=xhigh`, `enable_thinking=true`
Non-thinking sampling	`temperature=0.7`, `top_p=0.8`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`
Non-thinking mode	`enable_thinking=false`
Max new tokens	4,096
Context	native 262,144
Draft tokens	7 per verification step

Dataset	Samples	Thinking (`xhigh`) AL	Non-thinking AL
AA-LCR	100	3.8475	4.3578
AIME26	30	2.9922	5.2214
GSM8K	1,319	3.8272	5.5458
HumanEval	164	3.3457	6.5176
LiveCodeBench	1,055	2.7684	4.5785
LongBench-v2	402	3.4772	3.7079
MATH500	500	3.4836	5.7381
MBPP	257	3.2696	5.4883
MMSpec	600	3.1216	3.3236
MT-Bench	80	2.8316	3.5207
SWE-bench Pro	731	2.6905	3.8953

Accepted length is calculated from the raw server counters:

text

accepted_length = 1 + accepted_tokens / draft_calls

Architecture

Setting	Value
Draft layers	5
Auxiliary hidden-state IDs	`[6, 20, 34, 48, 62]`
Hidden size	5,120
FFN intermediate	17,408
Attention	32 Q heads / 8 KV heads
Head dimension	128
Attention window	sliding window 2,048
Draft attention	non-causal inside the draft block
RoPE theta	1e7
Norm / activation	RMSNorm eps 1e-6 / SiLU
Block	7 draft tokens, `sample_from_anchor=true`
Dynamic convolution	kernel 2, group 16
Candidate selector	rank 256, top-k 16
Draft vocabulary	248,320
Parameters	1,924,404,480

Training data

A decontaminated, redistributable subset of the training corpus is released as `DaoCloud/Qwen3.8-27B-Drafter-SFT`.

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with Qwen3.8-27B or a compatible quantized version:

bash

vllm serve Qwen/Qwen3.8-27B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --mm-encoder-tp-mode data \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/Qwen3.8-27B-DFlash2-Exp",
    "num_speculative_tokens": 7
  }'

This checkpoint uses the 7-query sample_from_anchor=true layout and currently requires vLLM PR #54154.

Checkpoint

The BF16 checkpoint contains 81 tensors and 1,924,404,480 parameters. It is released as a 3,848,817,920-byte (3.584 GiB) safetensors file.

File	SHA256
`model.safetensors`	`9e1aad9323c3ffecc35b1130e222476f722238233263e7ef9f08389bf1ff2a04`

Future work

—Adaptive verification: explore confidence-head-driven adaptive draft budgets for high-concurrency serving, including varlen GDN support (vLLM #51869).
—MoE drafter: explore sparsely activated drafter architectures for large-scale serving, increasing model capacity without proportionally increasing active compute.

License

Apache-2.0.