iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit-fp16

Voxtral Mini 4B Realtime 4bit (float16)

This is a 4-bit quantized, float16-base MLX conversion of mistralai/Voxtral-Mini-4B-Realtime-2602.

Which variant should you pick?

Only the non-quantized weights differ between the two repos (norms, biases, scales, some embeddings). The quantized mat-mul weights are bit-identical. Transcription output is byte-identical on a 20 s French clip at temperature 0 (verified locally).

Conversion

Source model:

• —mistralai/Voxtral-Mini-4B-Realtime-2602

Local conversion command:

python -m mlx_audio.convert \
  --hf-path mistralai/Voxtral-Mini-4B-Realtime-2602 \
  --mlx-path /path/to/Voxtral-Mini-4B-Realtime-2602-4bit-fp16 \
  --quantize \
  --q-group-size 64 \
  --q-bits 4 \
  --dtype float16 \
  --model-domain stt

Quantization config:

• —bits: 4
• —group size: 64
• —mode: affine
• —non-quant dtype: float16

Files

Only the MLX runtime artifacts needed for inference:

• —model.safetensors
• —model.safetensors.index.json
• —config.json
• —generation_config.json
• —params.json
• —processor_config.json
• —tekken.json

Usage

pip install "mlx-audio[stt]"

from mlx_audio.stt.utils import load_model

model = load_model("iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit-fp16")
result = model.generate("audio.wav")
print(result.text)

Notes

• —Base model license remains Apache 2.0.
• —On M3/M4, prefer the -4bit (bf16) repo; there is no speed benefit to fp16 there and bf16's wider exponent range is slightly more robust.
• —Transcription quality was verified identical to the bf16 variant at temperature=0 on a 20 s French parliamentary audio clip.