luiscalisto/Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit

Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit

MLX 4-bit quantisation of `mistralai/Mistral-Small-3.1-24B-Instruct-2503`, converted for use on Apple Silicon via `mlx-vlm`.

Source model

• —Repository: `mistralai/Mistral-Small-3.1-24B-Instruct-2503`
• —Release: 2025-03
• —Family: mistral
• —Origin: eu
• —Languages / coverage: 24+ languages incl. all major EU languages; vision-language model (image + text input)
• —License: apache-2.0 (inherited)

Notes from upstream

Mistral AI (France). Multimodal: mistral3 architecture with a Pixtral vision encoder. Converted with mlx-vlm (not mlx-lm) so the language model, vision tower and multimodalprojector are quantised consistently with current mlx-vlm loaders. Older community 4-bit uploads that predate the affine quantisation mode fail to load under mlx-vlm 0.5.0.

Conversion details

• —Tool: mlx-vlm 0.5.0
• —Quantisation: 4-bit (defaults from mlx_vlm.convert)
• —Converted on: 2026-05-22

Usage

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("luiscalisto/Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit")
config = load_config("luiscalisto/Mistral-Small-3.1-24B-Instruct-2503-MLX-4bit")

image = ["path/to/image.jpg"]
prompt = apply_chat_template(processor, config, "Describe this image.", num_images=len(image))
print(generate(model, processor, prompt, image, max_tokens=128, verbose=False))

License and attribution

This is a quantised redistribution of mistralai/Mistral-Small-3.1-24B-Instruct-2503. The original model and its license terms (apache-2.0) carry through unchanged. Please cite the upstream authors when using this model. See the source repository for the authoritative model card and citation.

Conversion provenance

Produced by `llm-mlx-conversions`, a small utility for publishing community MLX 4-bit quants of open-weight LLMs.