Datasetpublic

dispatchAI/quantization-guide

Quantization Guide Reference for choosing the right GGUF quantization level for mobile deployment. Q4_K_M is the recommended sweet spot — 40% of FP16 size, 92% quality. 🚀 dispatchAI

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes10downloads

rootguide.json

guide.json51 linesDownload Raw Back to root

1[2  {3    "level": "Q2_K",4    "size_vs_fp16": "25%",5    "quality": "75%",6    "use_case": "Ultra-low RAM (1GB devices)",7    "recommended": false8  },9  {10    "level": "Q3_K_M",11    "size_vs_fp16": "30%",12    "quality": "85%",13    "use_case": "Very constrained devices",14    "recommended": false15  },16  {17    "level": "Q4_K_M",18    "size_vs_fp16": "40%",19    "quality": "92%",20    "use_case": "Sweet spot for mobile",21    "recommended": true22  },23  {24    "level": "Q5_K_M",25    "size_vs_fp16": "50%",26    "quality": "95%",27    "use_case": "Quality-sensitive mobile",28    "recommended": true29  },30  {31    "level": "Q6_K",32    "size_vs_fp16": "60%",33    "quality": "97%",34    "use_case": "Near-lossless mobile",35    "recommended": false36  },37  {38    "level": "Q8_0",39    "size_vs_fp16": "70%",40    "quality": "98%",41    "use_case": "High-quality, smaller than FP16",42    "recommended": false43  },44  {45    "level": "F16",46    "size_vs_fp16": "100%",47    "quality": "100%",48    "use_case": "Reference / debugging",49    "recommended": false50  }51]