Modelpublic

modal-labs/GLM-5.3-NVFP4-DFlash

sourceHugging Faceotherupdated 2d agoView on Hugging Face

0likes391downloads

Model Card

GLM-5.3-NVFP4-DFlash

This repository contains a DFlash draft model for `modal-labs/GLM-5.3-NVFP4`. It is not a standalone language model. It is intended to be paired with the target model in a speculative decoding server.

DFlash uses a lightweight block diffusion draft model to propose multiple tokens in parallel. The target model verifies those proposals, improving serving throughput while preserving the target model's output distribution.

Quick Start

Serve with SGLang v0.5.19 or later:

bash

python -m sglang.launch_server \
  --model-path modal-labs/GLM-5.3-NVFP4 \
  --quantization modelopt_fp4 \
  --tp-size 4 \
  --trust-remote-code \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path modal-labs/GLM-5.3-NVFP4-DFlash \
  --speculative-dflash-block-size 8 \
  --speculative-draft-model-quantization unquant \
  --speculative-draft-attention-backend fa4 \
  --host 0.0.0.0 \
  --port 30000

Keep the draft model unquantized. Quantizing it lowers the accept length.

License

Distributed under Z.ai's GLM-5.3 License, inherited from the target model.

Citation

If you find DFlash useful, please cite the original paper:

bibtex

@article{chen2026dflash,
  title   = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
  author  = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  journal = {arXiv preprint arXiv:2602.06036},
  year    = {2026}
}