CoolFace
Modelpublic

bartowski/EveryoneLLM-7b-Gemma-Base-GGUF-old

sourceHugging Faceotherupdated 3y agoView on Hugging Face
0likes429downloads
Model Card

Llamacpp Quantizations of EveryoneLLM-7b-Gemma-Base

Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2354">b2354</a> for quantization.

Original model: https://huggingface.co/rombodawg/EveryoneLLM-7b-Gemma-Base

Download a file (not the whole branch) from below:

FilenameQuant typeFile SizeDescription
EveryoneLLM-7b-Gemma-Base-Q8_0.ggufQ8_09.07GBExtremely high quality, generally unneeded but max available quant.
EveryoneLLM-7b-Gemma-Base-Q6_K.ggufQ6_K7.01GBVery high quality, near perfect, recommended.
EveryoneLLM-7b-Gemma-Base-Q5_K_M.ggufQ5KM6.04GBHigh quality, very usable.
EveryoneLLM-7b-Gemma-Base-Q5_K_S.ggufQ5KS5.87GBHigh quality, very usable.
EveryoneLLM-7b-Gemma-Base-Q5_0.ggufQ5_05.87GBHigh quality, older format, generally not recommended.
EveryoneLLM-7b-Gemma-Base-Q4_K_M.ggufQ4KM5.12GBGood quality, similar to 4.25 bpw.
EveryoneLLM-7b-Gemma-Base-Q4_K_S.ggufQ4KS4.84GBSlightly lower quality with small space savings.
EveryoneLLM-7b-Gemma-Base-Q4_0.ggufQ4_04.80GBDecent quality, older format, generally not recommended.
EveryoneLLM-7b-Gemma-Base-Q3_K_L.ggufQ3KL4.40GBLower quality but usable, good for low RAM availability.
EveryoneLLM-7b-Gemma-Base-Q3_K_M.ggufQ3KM4.06GBEven lower quality.
EveryoneLLM-7b-Gemma-Base-Q3_K_S.ggufQ3KS3.67GBLow quality, not recommended.
EveryoneLLM-7b-Gemma-Base-Q2_K.ggufQ2_K3.09GBExtremely low quality, not recommended.

Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski