Modelpublic

andrijdavid/Voxtral-Mini-3B-2507-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes240downloads

Model Card

Voxtral-Mini-3B-2507-GGUF

GGUF builds of mistralai/Voxtral-Mini-3B-2507 for voxtral.cpp, a ggml-based C++ runtime for Metal / CUDA / Vulkan / CPU.

This is the offline Voxtral model similar to a Whisper-style audio encoder + Ministral-3B decoder that transcribes by generating text tokens and very fast for batch / long-form transcription.

Quants

File	Notes
`Q8_0`	near-lossless, largest
`Q6_K`, `Q5_K`, `Q5_1`, `Q5_0`	high quality
`Q4_K_M`	recommended best size/quality trade-off
`Q4_K`, `Q4_1`, `Q4_0`	smaller
`Q3_K`	smallest usable (some quality loss)

Q2_K is intentionally not provided as it degrades this model and produces unusable output.

Usage

bash

# build voxtral.cpp, then:
./tools/download_model.sh Q4_K_M --model mini
./build/voxtral --model models/voxtral-3b/Q4_K_M.gguf --audio clip.wav

Audio must be 16 kHz mono WAV. Long audio is transcribed in 30 s windows. GPU is auto-detected; force with --gpu metal|cuda|vulkan|none.

License

Apache 2.0, inherited from the base model.