taobao-mnn/Qwen3.5-2B-Dflash
Qwen3.5-2B DFlash Draft Model
A DFlash speculative-decoding draft model for Qwen3.5-2B .
<details open> <summary><b>English</b></summary>
What is DFlash?
DFlash is a lightweight speculative-decoding draft method. Instead of decoding one token at a time, the draft model proposes a whole block of candidate tokens in a single forward pass, conditioned on hidden states captured from several intermediate layers of the target model. The target model then verifies the block in parallel and accepts the longest correct prefix. This lets the system emit multiple tokens per target forward pass while keeping the output identical to greedy target decoding, which speeds up inference when decoding is memory-bandwidth bound.
Model
Files
model.safetensors— draft model weightsconfig.json— draft model config
Benchmark
Eval config
- Backend: sglang
- Decoding: greedy, temperature = 0, max_tokens = 2048
- Metric: accept length (AL) = Σ completiontokens / Σ verifyct (tokens emitted per target verify pass)
Per dataset
Overall
</details>
<details> <summary><b>中文</b></summary>
DFlash 是什么?
DFlash 是一种轻量级的投机解码(speculative decoding)草稿方法。草稿模型不再逐 token 解码,而是借助从目标模型多个中间层捕获的隐状态,在一次前向中一次性提出一整块(block)候选 token;随后目标模型并行校验该 block,并接受最长的正确前缀。因此在保持与目标模型贪心解码完全一致输出的前提下,每次目标前向可产出多个 token ,进而在访存瓶颈时提高推理速度。
模型信息
文件说明
model.safetensors— 草稿模型权重config.json— 草稿模型配置
评测结果
评测配置
- 后端: sglang
- 解码: greedy, 温度 = 0, max_tokens = 2048
- 指标: 接受长度 (AL) = Σ completiontokens / Σ verifyct(每次目标校验前向产出的 token 数)
分数据集
总览
</details>