Modelpublic

sh0wie/Qwen3.8-Flash-Next-MTP-Drafter-MLX-bf16

sourceHugging Faceotherupdated 24d agoView on Hugging Face

6likes4.6kdownloads

Model Card

Qwen3.8-Flash-Next MTP drafter

Qwen3.8-Flash-Next ships with a multi-token-prediction head that guesses the next few tokens in one pass; public MLX conversions drop those tensors, and this repo restores them as a standalone 4.9 GB drafter you attach at serve time. The full model then checks every guess, so the text you get is exactly the text you would have got without the drafter, only sooner.

Which pack it pairs with

Any Qwen3.8-Flash-Next target, whatever its expert count, because compatibility is architectural rather than weight-dependent. The pmlx tiered packs already bundle it, so you only download this repo when you are serving a checkpoint that does not carry one. The engine quantizes it to 8 bits at load, which matches the published 8-bit head to the last bit, drafts in 2.9 ms per round instead of 4.2, and saves 2.3 GB of RAM.

What it buys

M4 Max, 128 GB, single stream, temperature 0. "Warm server" means the HTTP server on the path you actually serve on; the benchmark script runs its own loop and reads plain decode lower, so compare rows only within one harness.

Target pack and prompt	Without the drafter	With it	Accepted per round	Harness and date
288-expert 4-bit, short prompt, fixed block 3	61.9 tok/s	69.2 tok/s	2.04	warm server, 2026-09-03
288-expert 4-bit, short prompt, fixed block 3	51.5 tok/s	64.9 tok/s, up 25.9 percent with a 95 percent interval of 15.4 to 36.4	1.94	benchmark script, 2026-09-03
512-expert 4-bit, short prompt, fixed block 3	56.5 tok/s	46.9 tok/s	not recorded	warm server, 2026-09-02

Acceptance falls as you cut resident experts: 2.04 accepted per round with 288 experts kept, 1.96 at 240, 1.53 at 200 (benchmark script, 2026-09-03).

Drafting until the head is unsure, rather than to a fixed depth, is the default and was worth another 7.8 percent on code prompts, 9.9 percent on edit prompts and 9.4 percent on agent transcripts over a fixed block of 3 on the 288-expert pack (2026-09-02).

Install and run

Install pmlx with pip install git+https://github.com/gethamster/pmlx, then:

bash

pmlx pull drafter
pmlx serve --model sh0wie/Qwen3.8-Flash-Next-REAP-288-MLX-4bit
pmlx

The server finds the drafter on disk and turns speculative decoding on by itself. pmlx serve --no-draft turns it off.

Not measured, and not recommended

—On the 512-expert tiered pack the drafter currently costs speed rather than saving it, because verifying against a split expert store costs about twice what it costs on a pruned pack.
—Past about 8,192 tokens of context, speculative decoding is slower than plain decoding on every pack measured, so the server falls back to plain decoding above that on its own.
—Below about 200 resident experts the accepted-tokens-per-round figure falls faster than plain decode does, so a speculative workload pays twice for a deep residency cut.

License

Qwen Community License 1.0, inherited from the base model; see LICENSE.

Provenance

The 31 mtp.* tensors of Qwen/Qwen3.8-Flash-Next, taken from the official bf16 release and repacked in the standalone drafter layout with no other change.