Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP

<p align="center"> <a href="https://qwen.ai/"><img src="qwen-logo.png" width="96" height="95" alt="Qwen"></a> </p>

<p align="center"> <img src="https://img.shields.io/badge/Qwen-Qwen3.8-615CED?style=for-the-badge&logo=qwen&logoColor=white" alt="Qwen Qwen3.8"> <img src="https://img.shields.io/badge/AppleSilicon-MLX-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Apple silicon MLX"> <img src="https://img.shields.io/badge/NativeMTP-Included-22C55E?style=for-the-badge" alt="Native MTP included"> <img src="https://img.shields.io/badge/Vontra-oQ8-6E56CF?style=for-the-badge&logo=huggingface&logoColor=white" alt="Vontra oQ8"> </p>

<h1 align="center">Qwen3.8 Flash Next — MLX oQ8 with native MTP</h1>

<p align="center"> A near-uniform 8-bit MLX conversion of <a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next">Qwen/Qwen3.8-Flash-Next</a>, quantised directly from the official BF16 checkpoint with its native MTP draft block preserved. </p>

<p align="center"> <a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next">Original model</a> · <a href="https://qwen.ai/blog?id=qwen3.8-flash-next">Qwen overview</a> · <a href="https://github.com/ml-explore/mlx-vlm">MLX-VLM</a> · <a href="LICENSE">Qwen Community License 1.0</a> </p>

About this conversion

This repository contains an oQ8 MLX conversion produced directly from Qwen's official BF16 weights. Every final tensor, including the native MTP head, was rebuilt from that checkpoint. Group size 32 supports the model's 160-wide hashed n-gram embedding tables.

The upstream tokenizer, chat template, vision processor, generation configuration, native MTP configuration, licence, and attribution are preserved.

[!IMPORTANT] This checkpoint requires an oMLX or MLX-VLM runtime with explicit qwen4_exp native-MTP support. Stock runtimes that do not construct the Qwen4Exp MTP module may reject the 76 MTP tensors during strict weight loading.

[!CAUTION] Do not attach a Qwen3.8 27B drafter to this model. Flash Next has different hidden dimensions and includes its own matching MTP block.

Download and use

hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP \
  --local-dir Qwen3.8-Flash-Next-MLX-oQ8-MTP

Add the downloaded directory to an MTP-capable oMLX installation, select the model, and enable native MTP in its model settings.

Apple M3 Studio performance

Validated in oMLX on an Apple M3 Studio using a deterministic 128-token chat generation:

Native MTP delivered an 83% throughput uplift on this measured prompt. The MTP-enabled run accepted 69 of 110 draft proposals (62.7%) and produced coherent, non-repetitive prose. Separate deterministic gates returned exactly hello, identified Paris correctly, and answered a basic arithmetic prompt with 5 both before and after enabling MTP.

The first request after loading includes model and kernel warm-up and is not used as a steady-state throughput figure. Results vary with prompt length, cache state, sampling settings, runtime version, and memory pressure.

Architecture

Qwen3.8 Flash Next combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, hashed bigram and trigram embeddings, and a native next-token prediction block for speculative decoding.

For upstream evaluations, intended use, limitations, safety guidance, and the complete architecture discussion, see the original model card.

Conversion and validation

• —The converter read the official BF16 checkpoint directly.
• —Structural validation checked all 3,747 indexed tensors and all 37 shards.
• —MTP validation confirmed one configured draft layer and 76 converted MTP tensor entries.
• —Live generation validation ran with MTP disabled and enabled, covering deterministic exact instruction, factual recall, arithmetic, coherent long generation, MTP telemetry, and cache rollback.

This is a community conversion, not an official Qwen release.

License and attribution

The upstream model is released under the Qwen Community License 1.0. The required licence text is included in this repository.

Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion and packaging are provided by Vontra.

<!-- vontra-chooser-start -->

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

Quick start and demo prompt

hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ8-MTP --local-dir ./models/Qwen3.8-Flash-Next-MLX-oQ8-MTP

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow Vontra for new Apple Silicon releases and fixes. <!-- vontra-chooser-end -->