jclyons52/Qwen3.8-27B-UD-Q2_K_XL-MLX-imatrix-direct

Qwen3.8-27B UD-Q2KXL — MLX imatrix-direct (native MLX, dynamic quant)

Native-MLX port of unsloth's UD-Q2_K_XL dynamic quantization for Qwen/Qwen3.8-27B. Runs the quantizer search natively in MLX's affine format (group size 64) and writes final MLX weights directly — no GGUF round-trip, no mlx_lm.convert re-quantization.

Perplexity (wikitext-2-raw first 32k tokens, 512-token windows):

Note: at 2 bits the gap to the GGUF widens (+1.95) because unsloth's 2-bit quants use non-linear IQ codebooks that MLX's affine kernels cannot represent (details).

Method

1. 1.Bit map from unsloth's Qwen3.8-27B-UD-Q2_K_XL.gguf (dynamic per-tensor bit allocation).
2. 2.Importance-weighted affine search per group using unsloth's published imatrix (per-column activation energies).
3. 3.Final MLX weights emitted directly (packed uint32 codes + bf16 scales/biases) — the emitted weights are exactly what the search chose.

Limitations

• —Affine vs IQ-codebook gap. MLX quantized matmul supports only uniform affine grids; unsloth's ≤3-bit GGUFs use non-linear IQ codebooks. The gap vs GGUF is inherent to MLX's current kernels, not the weights.
• —Text-only. The base checkpoint's visual tower is not included.
• —Eval scope: wikitext-2, 32k tokens only; downstream tasks uncharacterized.
• —Not for further training.

Calibration data

Importance weighting: unsloth's published imatrix_unsloth.gguf.

Credits

• —unsloth — dynamic bit maps + imatrix
• —mlx-lm — MLX runtime
• —Pipeline: jclyons52/ud2mlx

Built 2026-08-28. Base model license inherited from Qwen/Qwen3.8-27B.