Modelpublic

merterbak/Mistral-Small-3.1-24B-Instruct-2503-GGUF

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

1likes664downloads

Model Card

Mistral-Small-3.1-24B-Instruct-2503

Mistral-Small-3.1-24B-Instruct-2503 model available in multiple GGUF quantization formats, also stored on Xet for fast and efficient access. Available in following quantization levels:

Quantization Level	Bit Precision	Quantization Type
Q2_K	2-bit	K-means clustering
Q3KS	3-bit	Small k-means clustering
Q3KM	3-bit	Medium k-means clustering
Q3KL	3-bit	Large k-means clustering
Q4_0	4-bit	Standard quantization
Q4_1	4-bit	Standard quantization
Q4KS	4-bit	Small k-means clustering
Q4KM	4-bit	Medium k-means clustering
Q5_0	5-bit	Standard quantization
Q5_1	5-bit	Standard quantization
Q5KS	5-bit	Small k-means clustering
Q5KM	5-bit	Medium k-means clustering
Q6_K	6-bit	K-means clustering
Q8_0	8-bit	Standard quantization