Akicou/Qwen3.8-27B-DFlash2-GGUF

Qwen3.8-27B-DFlash2 GGUF

GGUF conversions of `z-lab/Qwen3.8-27B-DFlash2`, the DFlash 2 draft model for `Qwen/Qwen3.8-27B`.

This is not a standalone language model. It is a draft model for speculative decoding: it drafts blocks of tokens that Qwen3.8-27B then verifies. Loading it on its own fails with dflash requires ctx_other to be set, which is the expected response for a draft model without its target. Run it next to the target model with --spec-type draft-dflash.

DFlash 2 predicts a whole block of tokens in one pass and keeps the top candidates at every position. A selector then traces one path through the candidates. Two-tap dynamic convolutions in the backbone keep draft quality from dropping toward the end of the block. Decoding is lossless: greedy output matches the target model, and sampling preserves its distribution. The z-lab repository is a mirror of `incoai/Qwen3.8-27B-DFlash2`. The DFlash 2 paper and reference code are at inco.ai/blog/dflash2 and github.com/z-lab/dflash.

Files

There is no Q6KM in llama.cpp; Q6K is the 6-bit K-quant. If you want the smallest usable draft, take Q4KS. If you want the draft to cost as little accuracy as possible, take Q6K or BF16.

Model info

Source model: z-lab/Qwen3.8-27B-DFlash2, commit 50307d4c4cde6860d4eee73e2547cd786fe8e8a4, Apache-2.0.

Requirements

DFlash 2 support in llama.cpp is not merged into master yet (as of 19 August 2026). Use a build from PR #27342 or a later master commit once it lands. The files here were produced and validated with a build of that PR merged at commit 5c094f254. Official release binaries will reject the dflash architecture keys until the PR ships.

Usage with llama.cpp

Serve the target model with this draft (DFlash 2 is detected from the checkpoint, no extra flag needed):

./llama-server -m Qwen3.8-27B-Q4_K_M.gguf \
    -md Qwen3.8-27B-DFlash2-Q6_K.gguf \
    --spec-type draft-dflash --spec-draft-n-max 7 -fa on --jinja

The --spec-draft-n-max 7 matches the block size of 8, which gives 7 draft tokens per verification step. Generation without a server:

./llama-cli -m Qwen3.8-27B-Q4_K_M.gguf \
    -md Qwen3.8-27B-DFlash2-Q6_K.gguf \
    --spec-type draft-dflash --spec-draft-n-max 7 -fa on --jinja \
    -p "Hello" -n 128

The draft model needs roughly 1.0 to 3.6 GiB of memory depending on the quant, on top of the target model.

How these files were made

# tokenizer comes from the target model, so --target-model-dir is required
python convert_hf_to_gguf.py z-lab/Qwen3.8-27B-DFlash2 \
    --target-model-dir Qwen/Qwen3.8-27B \
    --outtype bf16 --outfile Qwen3.8-27B-DFlash2-BF16.gguf

./llama-quantize Qwen3.8-27B-DFlash2-BF16.gguf Qwen3.8-27B-DFlash2-Q6_K.gguf Q6_K
./llama-quantize Qwen3.8-27B-DFlash2-BF16.gguf Qwen3.8-27B-DFlash2-Q4_K_M.gguf Q4_K_M
./llama-quantize Qwen3.8-27B-DFlash2-BF16.gguf Qwen3.8-27B-DFlash2-Q4_K_S.gguf Q4_K_S

All 81 tensors load in every file. The quantizer reports these figures:

What DFlash 2 buys you

Numbers below come from the z-lab model card. They measured SGLang on one NVIDIA H200, FlashAttention 3, block size 8, temperature 1.0, top-p 0.95, top-k 20. Acceptance length is the per-request mean of completion tokens divided by verification steps; higher is better. All speculative methods propose 7 tokens per step.

At concurrency 1, throughput speedup over plain autoregressive decoding ranged from 2.67x to 3.43x across those five tasks. The llama.cpp PR reports 5.46 acceptance on GSM8K at Q4KM on an Apple M5 Pro with 64 GB, using the first 8 GSM8K problems.

Sources

• —Draft model: `z-lab/Qwen3.8-27B-DFlash2` (mirror of incoai/Qwen3.8-27B-DFlash2)
• —Target model: `Qwen/Qwen3.8-27B`
• —llama.cpp DFlash 2 support: PR #27342
• —DFlash 2 blog: inco.ai/blog/dflash2, code: github.com/z-lab/dflash

License: Apache-2.0, same as the source model. The tokenizer in these GGUFs comes from the target model, Qwen/Qwen3.8-27B, also Apache-2.0.