petergilani/Qwen3-Coder-Next-8bit-g128

Model Card for Qwen3-Coder-Next-8bit-g128

Quantized Qwen/Qwen3-Coder-Next using mlx-lm to 8-bit with groupsize 128 for main weights and fine-grained groupsize 64 for MoE weights, with the aim of maximum accuracy for 8-bit quantization.

Updated Evaluation Results (February 13, 2026)

Comprehensive evaluation results from thorough testing using mlxlm.evaluate with mmlupro (200 questions per domain, numshots=1, temp=1.0, topp=0.95, top_k=40, seed=123):

Direct Comparison Summary (8-bit g64 vs g128)

Average Performance

• —8-bit g64: 87.0% average
• —8-bit g128 (this model): 87.0% average
• —Difference: 0.0 percentage points (virtually identical)

Key Observations

• —Both 8-bit models show nearly identical average performance (87.0%)
• —Performance differences are domain-specific rather than model-specific
• —8-bit g64 performs better in Math, Engineering, and Physics
• —8-bit g128 (this model) performs better in Computer Science and Chemistry, with Computer Science being the most significant improvement (+2.5%)

Memory Usage Comparison

• —8-bit g64:
• —Peak memory usage: 95.374 GB (Math) to 100.740 GB (Engineering)
• —Average memory usage: ~98.6 GB

• —8-bit g128 (this model):
• —Peak memory usage: 97.702 GB (Computer Science) to 98.510 GB (Engineering)
• —Average memory usage: ~97.8 GB

Full Quantization Spectrum Comparison (Updated)

Original Evaluation Results

Testing with mlxlm.evaluate using mmlupro with 50 Qs for each topic, comparing the 8-bit g128 quant with the 8-bit g64 quant:

Average Performance

• —8-bit g64: 84.4% average
• —8-bit g128: 88.0% average
• —Improvement: +3.6 percentage points with group size 128 (main weights) with fine-grained 64 (MoE weights)

Key Benefits

• —The g128 group_size applied to main weights (with fine-grained g64 for MoE weights) is also ~2GB smaller than the g64 version
• —At the time of uploading, there were not any 8bit g128 quants of this model available on HF
• —Also tried MXFP8 but found equal or lower accuracy than g64 for all topics (Engineering dropped to 0.76 and Chemistry dropped to 0.88 (-4% and -2% compared to g64))

Model Details

• —Base Model: Qwen/Qwen3-Coder-Next
• —Library: mlx-lm
• —Quantization: 8-bit with groupsize 128 for main weights and groupsize 64 for MoE weights
• —License: apache-2.0
• —Pipeline Tag: text-generation

Usage

import mlx_lm
from mlx_lm.sample_utils import make_sampler

model_path = "petergilani/qwen3-coder-next-8bit-g128"
model, tokenizer = mlx_lm.load(model_path)

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=40)

prompt = "Write a Python function to calculate the factorial of a number."
response = mlx_lm.generate(
    model, 
    tokenizer, 
    prompt=prompt,
    sampler=sampler,
    max_tokens=512
)
print(response)