mlx-community/Qwen3.8-Flash-Next-Uncensored-oQ5e-mtp

Qwen3.8-Flash-Next-Uncensored (MLX, oQ5e, MTP)

Calibrated 5-bit mixed-precision MLX quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored, the BF16 abliterated build of Qwen/Qwen3.8-Flash-Next. Produced with oMLX oQe (oQ level 5 with imatrix weighting). 5.72 bits per weight effective, 128.54 GB on disk (119.7 GiB) across 25 safetensors shards. Everything from the source is kept: the routed experts, the 51B-parameter n-gram embedding table (quantized, not left in BF16), the Lightning MTP head, and the full vision + video tower.

This is the build that runs the model's full 262,144-token context on a 128 GB Apple Silicon Mac. A real 250,073-token request completed in 214 s on a MacBook Pro M5 Max with Lightning MTP and TurboQuant KV on (numbers below).

[!WARNING] The source model has had its safety alignment removed by abliteration. It will comply with requests the original Qwen3.8-Flash-Next refuses. Released for research, red-teaming and interpretability work. Read the source model card's disclaimer before use and add your own moderation if you deploy it. You are responsible for what you do with it.

Model details

What is in this repo

The size shown by oMLX's estimator and in its upload dialog (119.7 "GB") is GiB. The decimal figure is 128.5 GB.

Quantization layout

oQ is sensitivity-driven mixed precision: every layer's quantization error is measured on calibration data, and bits go where the data says they matter. The "e" (oQe) adds an importance matrix so that inside each quantization group the clipping favors the weights that carry activation energy. Output is plain affine quantization that any MLX runtime can read.

Bit histogram over the 871 quantized text tensors: 209 at 8-bit, 498 at 6-bit, 164 at 5-bit. 128 n-gram shards at 5-bit. The exact per-tensor map is in config.json under quantization.

How it was made

Built on a MacBook Pro M5 Max (128 GB) with oMLX 0.7.0.dev2 from the 360 GB BF16 source, streamed tensor by tensor (the full model never sits in memory).

• —Sensitivity pass: measured on a uniform 4-bit MLX build of the same model (the orcarouter 4-bit release), since the BF16 source is 2.8x the machine's memory. 48 layers scored, 128 samples x 256 tokens.
• —Importance matrix: collected on a temporary uniform 4-bit proxy that oMLX builds and deletes itself. Calibration set oqe_code_multilingual (code, English, Korean, Chinese, Japanese, tool calls, reasoning). The adaptive collector ran to its maximum of 1,024 samples x 512 tokens: 937 tensor entries, 6,144 layer forwards, 128 MTP-head forwards. Expert coverage 75,243 of 75,264 (21 experts were never routed on the calibration set and fall back to plain oQ). Three tensors have no imatrix entry by design (lm_head and the two hyper-connection mixer projections) and are quantized with plain oQ. Full report in oq_imatrix_report.json.
• —Time: about 80 minutes end to end, of which 46 minutes was imatrix collection. The imatrix cache is keyed by source, calibration set and sample counts, not by bit width, so the same matrix was reused unchanged for the sibling oQ6e build.

Requirements

Apple Silicon only. Tested with oMLX 0.7.0.dev2, which ships its own qwen4_exp implementation (Lightning MTP, the PLE table on SSD, TurboQuant KV). The checkpoint is standard MLX safetensors, but stock mlx-lm / mlx-vlm need qwen4_exp support to load it and were not tested here.

Disk: 129 GB for the model, plus room for oMLX's KV cache spillover.

Running it with oMLX

Drop the folder into ~/.omlx/models/ (or download it there) and the model is discovered on the next rescan. Settings used for the numbers below, on a 128 GB M5 Max:

Any OpenAI-compatible client then works against http://localhost:8000/v1. Vision input goes in user messages as image_url parts.

Measured performance

MacBook Pro M5 Max, 128 GB, 40-core GPU, oMLX 0.7.0.dev2, n-gram table on SSD, Lightning MTP with 3 drafts, TurboQuant KV 4-bit unless noted. Single request.

Prefill (prompt processing)

The 250k request ran with no memory-guard throttling; every prefill chunk stayed at the full 2,048-token step. Without TurboQuant KV the prefill numbers are 5 to 8 percent higher.

Decode (generation)

Decode is bound by the n-gram gathers from SSD in this configuration. On a machine where the table is resident, expect materially faster decode; oMLX's own estimate for the SSD path is about 2.5x slower than resident.

Compared with the sibling builds

oQ5e is the highest level that keeps the full context window on a 128 GB machine. oQ6e trades that for 6-bit experts; see its card for the exact limits.

Limitations

• —Apple Silicon and MLX only.
• —Safety alignment removed by the source model's abliteration. No guardrails.
• —21 of 75,264 routed experts and 3 shell tensors were quantized without imatrix weighting (plain oQ), as noted above.
• —The n-gram table runs from SSD on 128 GB machines, which caps decode speed; this is a property of the machine, not the quantization.
• —No formal perplexity or benchmark evaluation was run on this quant. The measurements above are throughput and a correctness spot check (long-context retrieval answer, tool calls and vision were exercised through oMLX).

Credits

• —Base model: Qwen (Qwen3.8-Flash-Next).
• —Abliterated source: OrcaRouter.
• —Quantization tooling: oMLX by jundot, oQ specification.

@misc{omlx,
  title  = {oMLX: LLM inference optimized for Apple Silicon},
  author = {jundot},
  year   = {2026},
  url    = {https://github.com/jundot/omlx}
}