Modelpublic

programmer-666/Qwen3-Coder-Next-oQ8e

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

2likes197downloads

Model Card

Qwen3-Coder-Next-oQ8e

An 8-bit-equivalent quantization of Qwen/Qwen3-Coder-Next, produced with oMLX and intended for local inference on Apple Silicon.

Why This Quant Exists

Qwen3-Coder-Next is not the newest coding model on the block anymore, but it remains a reliable, well-behaved workhorse for agentic coding tasks: strong tool use, long-context stability, and a favorable 3B-active / 80B-total parameter ratio that keeps it fast on consumer hardware. Rather than let a still-useful model sit on outdated quantization, it was re-quantized here with a current oMLX quantization pipeline (oQ8e) to keep it fully usable with up-to-date MLX tooling and to serve as a high-fidelity reference point against lower-bit quants.

Model Details

—Base model: Qwen/Qwen3-Coder-Next (qwen3_next architecture, 80B total / 3B active parameters, 256k native context)
—Quantization method: oMLX, 8-bit-equivalent (oQ8e)
—License: Apache 2.0 (inherited from base model)
—Format: MLX

Usage

python

from mlx_lm import load, generate

model, tokenizer = load("programmer-666/Qwen3-Coder-Next-oQ8e")

prompt = "Write a quick sort algorithm."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=text, max_tokens=2048)
print(response)

The model can also be served through oMLX's OpenAI-compatible API endpoint for use with agentic coding tools.

Benchmarks

All benchmarks were run with oMLX. Four variants were tested: this model (oQ8e) and an oQ2.7e quant, each in a default and an "Adjusted" configuration. Tests were performed on a MacBook Pro M4 Max ARM processor and 128 GB of memory.

The "Adjusted" columns were run with tuned oMLX serving parameters (context window, sampling, and related runtime settings), rather than the server's default configuration.

Prompt Processing Speed (tokens/s)

Context	oQ8e	oQ8e-Adjusted	oQ2.7e	oQ2.7e-Adjusted
1,024	831.65	1232.10	125.90	1262.70
4,096	583.70	1313.25	1088.10	1334.30
8,192	562.50	1179.90	1109.80	1178.20
16,384	599.20	975.95	991.40	1021.10
32,768	534.90	847.60	838.70	801.20
65,536	446.70	594.85	537.50	476.80
131,072	252.20	342.30	290.70	350.90
200,000	188.90	259.40	219.40	222.30

Generation Speed (tokens/s)

Context	oQ8e	oQ8e-Adjusted	oQ2.7e	oQ2.7e-Adjusted
1,024	51.80	66.25	17.90	71.20
4,096	37.90	64.45	67.40	66.80
8,192	36.10	62.65	67.30	64.60
16,384	52.20	59.35	62.20	62.00
32,768	36.90	55.60	59.90	56.90
65,536	32.20	35.25	30.00	31.30
131,072	23.60	30.40	28.30	30.90
200,000	20.60	24.35	23.20	20.40

Peak Memory (GB)

Context	oQ8e	oQ8e-Adjusted	oQ2.7e	oQ2.7e-Adjusted
1,024	80.16	80.16	70.05	31.45
4,096	80.92	80.92	66.92	32.21
8,192	81.17	81.17	66.02	32.45
16,384	81.44	81.50	32.73	32.61
32,768	81.94	82.06	33.34	33.22
65,536	82.69	82.72	33.85	34.33
131,072	83.98	83.98	35.15	35.86
200,000	85.78	85.72	36.94	36.83

All runs used tg128 (128 generated tokens) at each listed prompt length.

Inference Parameters

Benchmarks and general usage were run with the following oMLX serving configuration:

Parameter	Value
Reasoning Parser	qwen3coder
Context Window	262,144
Max Tokens	65,536
Temperature	1
Top P	0.95
Top K	40
Min P	0
Repetition Penalty	1
Presence Penalty	0
TTL	3,600s (global default)

Notes

—The oQ8e quant trades memory footprint (roughly 80 to 86 GB peak) for accuracy closer to the original weights, while oQ2.7e trims memory substantially at the cost of some quality.
—Throughput at very long context (131k+) drops sharply for all variants, which is expected given the attention cost of long-context prefill.

Acknowledgments

Thanks to the Qwen team for the base model and to the MLX / oMLX community for the tooling used to produce this quant.