Modelpublic

Abiray/gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-GGUF

sourceHugging Faceupdated 6mo agoView on Hugging Face

9likes928downloads

Model Card

gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-GGUF

This repository contains GGUF format model files for Ayodele01's gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill.

These models were compiled and quantized via llama.cpp to enable efficient local inference on consumer hardware.

Available Quantizations

File Name	Description
`gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q8_0.gguf`	8-bit quantization. Near unquantized performance, largest file size.
`gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q6_K.gguf`	6-bit quantization. Very high quality, minimal degradation from original.
`gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q5_K_M.gguf`	5-bit quantization. Higher quality, slightly larger size and slower inference.
`gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q4_K_M.gguf`	4-bit quantization. Recommended. Excellent balance of speed, memory usage, and quality.
`gemma-4-E4B-Gemini-3.1-Pro-Reasoning-Distill-Q3_K_M.gguf`	3-bit quantization. Very high compression, fast inference, lower quality.