Modelpublic

modal-labs/GLM-5.3-Flash-DFlash

sourceHugging Facemitupdated 1d agoView on Hugging Face

0likes378downloads

Model Card

GLM-5.3-Flash-DFlash

This repository contains a DFlash draft model for `zai-org/GLM-5.3-Flash`. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.

DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.

Quick Start

GLM-5.3-Flash needs the DFlash capture hooks in the glm5_next model, available on SGLang main. An example deployment is:

bash

python -m sglang.launch_server \
  --model-path zai-org/GLM-5.3-Flash \
  --tp-size 4 \
  --trust-remote-code \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path modal-labs/GLM-5.3-Flash-DFlash \
  --speculative-dflash-block-size 8 \
  --speculative-draft-model-quantization unquant \
  --speculative-draft-attention-backend trtllm_mha \
  --speculative-draft-kv-cache-dtype fp8_e4m3 \
  --host 0.0.0.0 \
  --port 30000

Keep the draft model unquantized. Quantizing it lowers the accept length.

License

Distributed under the MIT License, inherited from the target model.

Citation

If you find DFlash useful, please cite the original paper:

bibtex

@article{chen2026dflash,
  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026}
}