petergilani/Qwen3-Coder-Next-4bit-g128

Model Card for petergilani/Qwen3-Coder-Next-4bit-g128

Quantized Qwen/Qwen3-Coder-Next using mlx-lm to 4-bit with group_size 128 for main weights, with the aim of maximum efficiency for 4-bit quantization.

Updated Evaluation Results (February 13, 2026)

Comprehensive evaluation results from thorough testing using mlxlm.evaluate with mmlupro (200 questions per domain, numshots=1, temp=1.0, topp=0.95, top_k=40, seed=123):

Direct Comparison Summary (4-bit g64 vs g128)

Average Performance

• —4-bit g64: 84.1% average
• —4-bit g128 (this model): 84.2% average
• —Difference: 0.1 percentage points (virtually identical)

Key Observations

• —Both 4-bit models show nearly identical average performance (84.1-84.2%)
• —Performance differences are domain-specific rather than model-specific
• —4-bit g128 (this model) performs better in Physics and Engineering
• —4-bit g64 performs slightly better in Math and Chemistry

Memory Usage Comparison

• —4-bit g64:
• —Peak memory usage: 56.355 GB (Math) to 61.010 GB (Physics)
• —Average memory usage: ~60.0 GB

• —4-bit g128 (this model):
• —Peak memory usage: 51.989 GB (Math) to 59.084 GB (Engineering)
• —Average memory usage: ~57.3 GB

Full Quantization Spectrum Comparison (Updated)

Original Evaluation Results

Testing with mlxlm.evaluate using mmlupro with 50 Qs for each topic, comparing the 4-bit g128 quant with the 4-bit g64 quant:

Average Performance

• —4-bit g64: 84.4% average
• —4-bit g128: 86.0% average
• —Improvement: +1.6 percentage points with group size 128

Key Benefits

• —The g128 group_size applied to main weights provides modest memory savings (~2.7 GB less on average)
• —Performance differences between group sizes are minimal for 4-bit quantization
• —4-bit quantization provides significant memory efficiency (~40% less than 8-bit models)
• —Thermal characteristics are highly favorable compared to higher-bit quantizations, making this suitable for thermal-constrained environments

Model Details

• —Base Model: Qwen/Qwen3-Coder-Next
• —Library: mlx-lm
• —Quantization: 4-bit with group_size 128 for main weights
• —License: apache-2.0
• —Pipeline Tag: text-generation

Usage

import mlx_lm
from mlx_lm.sample_utils import make_sampler

model_path = "petergilani/Qwen3-Coder-Next-4bit"
model, tokenizer = mlx_lm.load(model_path)

sampler = make_sampler(temp=1.0, top_p=0.95, top_k=40)

prompt = "Write a Python function to calculate the factorial of a number."
response = mlx_lm.generate(
    model, 
    tokenizer, 
    prompt=prompt,
    sampler=sampler,
    max_tokens=512
)
print(response)