an0nya/Ministral-3-14B-Reasoning-2512-oQ4e

Ministral-3-14B-Reasoning-2512-oQ4e (MLX, imatrix-weighted mixed precision)

A mixed-precision MLX quantization of mistralai/Ministral-3-14B-Reasoning-2512, produced with oMLX's oQ4e pipeline on a 16 GB M4 Mac Mini.

7.94 GB · 2 shards · text-only (vision tower removed) · base group_size=64, bits=4, mode=affine with per-tensor bit allocation driven by measured layer sensitivity.

Standard mlx-lm safetensors — usable in any MLX runtime.

What makes this one unusual

Two of the three inputs to this quantization were supplied from outside the tool that built it. That is the interesting part, and it's why the provenance is spelled out in detail below rather than summarised as "imatrix quant."

1. The importance matrix was transplanted from a llama.cpp GGUF

oMLX collects its own importance matrix by running calibration forward passes over the unquantized model. On this machine that meant a 26 GB footprint against a ~7.4 GB budget, which fails — the quantization itself streams tensor-by-tensor at ~3–4 GB, but calibration does not.

Instead, imatrix_unsloth.gguf_file from unsloth/Ministral-3-14B-Reasoning-2512-GGUF was converted into oMLX's native oQe cache format. This works because llama.cpp's GGUF imatrix and oMLX's oQe cache store the same statistic under the same key names (.in_sum2 / .counts) — summed squared input activations per channel. The conversion is a tensor-name remap, not a reinterpretation.

Thanks to [unsloth](https://huggingface.co/unsloth) — their calibration corpus is doing real work here. It is 139 chunks × 5120 tokens ≈ 722k tokens, roughly 10.9× the 128 × 512 = 65k tokens oMLX would have collected on its own, and generated per-model rather than from a fixed generic corpus.

2. The layer-sensitivity map was measured externally

oMLX's sensitivity measurement fails on every mistral3 checkpoint:

_forward_layer: all signatures failed for TransformerBlock:
    TransformerBlock.__call__() missing 1 required positional argument: 'attn_scale'

mlx_vlm's Mistral3 TransformerBlock takes Llama-4-style attention temperature scaling as positional argument #2; oMLX's forward-signature probe doesn't pass it, so every layer returns None and the map comes back empty. This is an upstream bug, not a property of this model.

The map here was measured with a small script that imports oMLX's own calibration loader, layer discovery, perturbation-width selection and scoring, overriding only the broken forward call. Scoring is oMLX's verbatim: relative MSE from re-quantizing each layer one valid bit-width down, computed in float32.

The resulting curve is asymmetric — a sharp three-layer entry spike, a steep crash, then a near-monotonic climb to the output:

L0-L2    0.01648      entry spike
L3       0.000089     minimum (185x below L2)
L8-L15   0.00089
L24-L31  0.00265
L32-L37  0.00618
L38-L39  0.01858      output ramp

So the most compressible layers are early (L3–L15), not mid-stack, while L0–L2 and the final ~5 layers carry the cost. 32 of the 36 steps from L3 to L39 are increases. A 4×128 smoke run and the full 128×256 run independently agreed on the top-5 most-sensitive ordering.

3. Vision weights were stripped

The source is a VLM (mistral3 with a Pixtral encoder). This build is text-only — the vision tower is excluded. If you need image input, use the original model.

Verification

From oq_imatrix_report.json, shipped in this repo:

applied:      280   (40 layers × 7 tensors: q/k/v/o/gate/up/down)
mismatched:   0
missing:      2     (lm_head, embed_tokens)

Zero mismatches — every mapped tensor name resolved against the checkpoint. The two "missing" entries are expected: llama.cpp imatrices don't carry token_embd/output entries, and those tensors get oMLX's standard (non-imatrix) treatment at 8-bit.

provenance/ — everything needed to reproduce this

Note on the sidecar: its num_samples / seq_length fields (128 / 512) are an oMLX cache key, chosen to match what oMLX passes at runtime. They do not describe how the matrix was collected — the real figures are in the provenance block (139 chunks × 5120 tokens). This is stated in the file itself rather than left to be inferred.

What is NOT established

No perplexity or benchmark comparison has been run against a flat 4-bit quantization of the same model. The imatrix was demonstrably applied; whether it demonstrably helps is untested here. Treat any quality claim accordingly.

The one anecdote available: on a symbolic-communication task where a flat 4-bit MLX quant of this model injected stray whitespace into 4 of 11 emitted symbol strings, this build emitted 0 of 11. That is a single unmatched run, the two runs' agents chose different message lengths, and the underlying cause is tokenizer behaviour around rare glyphs rather than quantization. It is not evidence, only the reason the comparison seemed worth mentioning.

Usage

from mlx_lm import load, generate

model, tokenizer = load("an0nya/Ministral-3-14B-Reasoning-2512-oQ4e")
print(generate(model, tokenizer, prompt="...", max_tokens=512))

This is a reasoning model — it emits a thinking block before its answer. Budget tokens accordingly; short prompts can still produce long deliberations.

Build details