CoolFace
Modelpublic

Vontra/Qwen3.8-Flash-Next-MLX-oQ4

sourceHugging Faceotherupdated 16d agoView on Hugging Face
1likes4.2kdownloads
Model Card

<p align="center"> <img src="https://img.shields.io/badge/Qwen-Qwen3.8-615CED?style=for-the-badge&logo=qwen&logoColor=white" alt="Qwen Qwen3.8"> <img src="https://img.shields.io/badge/Apple_Silicon-MLX-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Apple silicon MLX"> <img src="https://img.shields.io/badge/Vontra-oQ-6E56CF?style=for-the-badge&logo=huggingface&logoColor=white" alt="Vontra oQ"> </p>

<h1 align="center">Qwen3.8 Flash Next — MLX oQ4</h1>

<p align="center"> A mixed-precision MLX conversion of <a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next">Qwen/Qwen3.8-Flash-Next</a>, quantised directly from the official BF16 checkpoint. </p>

<p align="center"> <a href="https://huggingface.co/Qwen/Qwen3.8-Flash-Next">Original model</a> · <a href="https://qwen.ai/blog?id=qwen3.8-flash-next">Qwen overview</a> · <a href="https://github.com/ml-explore/mlx-vlm">MLX-VLM</a> · <a href="LICENSE">Qwen Community License 1.0</a> </p>

About this conversion

This repository contains an oQ4 mixed-precision MLX conversion produced directly from Qwen's BF16 weights. Layer sensitivity was measured with a validated 4-bit proxy; final tensor quantisation reread the official BF16 checkpoint. Group size 32 supports the model's 160-wide hashed n-gram embedding tables.

ItemValue
Base model`Qwen/Qwen3.8-Flash-Next`
FormatMLX safetensors
QuantisationoQ4 mixed precision; 4-bit affine base with protected layers at 5/8-bit
Base group size32
Weight shards22
Weight size111.69 GB (104.02 GiB)
Configured context262,144 tokens
Architectureqwen4_exp vision-language sparse MoE

The upstream tokenizer, chat template, vision processor, and generation configuration are preserved. The optional upstream MTP head is not included.

[!IMPORTANT] Qwen3.8 Flash Next uses the new qwen4_exp architecture. Use an oMLX or MLX-VLM build that explicitly lists qwen4_exp support. Older MLX-VLM releases cannot load this checkpoint.
[!CAUTION] Do not attach a Qwen3.8 27B MTP drafter to this model. The hidden sizes differ and the drafter is incompatible with Flash Next.

Quick start

bash
hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ4 \
  --local-dir Qwen3.8-Flash-Next-MLX-oQ4

With a compatible MLX-VLM runtime:

bash
python -m mlx_vlm.generate \
  --model Qwen3.8-Flash-Next-MLX-oQ4 \
  --prompt "Explain sparse mixture-of-experts routing." \
  --max-tokens 512

Measured performance

Validated on an Apple M3 Studio with deterministic text-only generation after model load:

Test pathResult
Standalone MLX exact-copy generation30.4 tokens/s
Standalone MLX, 139-token explanatory response27.9 tokens/s
oMLX server, warmed 156-token response27.9 tokens/s
oMLX server, warmed 1,044-token response27.0 tokens/s

Prompt processing measured 27.2–32.2 tokens/s in the standalone short tests. The first oMLX server request reported 6.72 seconds to load the model; that one-off load time is separate from generation speed. The longer warmed server run is the most representative sustained result. Results vary with prompt length, cache state, sampling settings, runtime version, and memory pressure.

Architecture

Qwen3.8 Flash Next combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, and hashed bigram/trigram embeddings.

Architecture detailUpstream value
Language-model parameters125B total / 6B active
N-gram embedding51B parameters
Layers48
Routed / active experts512 / 10, plus 1 shared
Attention heads / KV heads24 / 2
Hidden size2,560
Native configured context262,144 tokens

For upstream evaluations, intended use, limitations, safety guidance, and the complete architecture discussion, see the original model card.

Conversion and validation

  • —Source: official BF16 checkpoint.
  • —Sensitivity-guided mixed-precision allocation: 4-bit base with 228 protected modules at 5/8-bit.
  • —All 3,671 converted tensors and 22 indexed shards were checked locally.
  • —Deterministic exact-copy and explanatory-generation tests passed on Apple silicon.
  • —The release payload was scanned for credentials, personal contact details, private paths, private network information, logs, caches, and private organisation data.

This is a community conversion, not an official Qwen release.

License and attribution

The upstream model is released under the Qwen Community License 1.0. The required licence text is included in this repository.

Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion, Apple-silicon validation, and packaging are provided by Vontra.

<!-- vontra-chooser-start -->

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

An unresolved long-generation repetition report involves oQ4-MTP; this update does not establish a fix.

Quick start and demo prompt

bash
hf download Vontra/Qwen3.8-Flash-Next-MLX-oQ4 --local-dir ./models/Qwen3.8-Flash-Next-MLX-oQ4

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

text
Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow Vontra for new Apple Silicon releases and fixes. <!-- vontra-chooser-end -->