CoolFace
Modelpublic

kirilldual0987/MIXdevAI-yandexGPT5-8B-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes603downloads
Model Card

MIXdevAI-yandexGPT5-8B-GGUF

Full set of GGUF quantizations for Kolyadual/MIXdevAI-yandexGPT5-8B.

Files

FileQuantizationType
mixdev-ya-f16.ggufF16Base Model
mixdev-ya-q8_0.ggufQ8_0Text Model
mixdev-ya-q6_k.ggufQ6_KText Model
mixdev-ya-q5_k_m.ggufQ5KMText Model
mixdev-ya-q5_k_s.ggufQ5KSText Model
mixdev-ya-q5_1.ggufQ5_1Text Model
mixdev-ya-q5_0.ggufQ5_0Text Model
mixdev-ya-q4_k_m.ggufQ4KMText Model
mixdev-ya-q4_k_s.ggufQ4KSText Model
mixdev-ya-q4_1.ggufQ4_1Text Model
mixdev-ya-q4_0.ggufQ4_0Text Model
mixdev-ya-q3_k_l.ggufQ3KLText Model
mixdev-ya-q3_k_m.ggufQ3KMText Model
mixdev-ya-q3_k_s.ggufQ3KSText Model
mixdev-ya-q2_k.ggufQ2_KText Model

Quantization notes

  • —Q8_0 / F16: Almost lossless. Best quality, largest size.
  • —Q6_K / Q5_K_M: Excellent balance between quality and size.
  • —Q4_K_M: Golden standard for local inference.
  • —Q3_K_M / Q2_K: Noticeable quality degradation. Use only if you have severe RAM/VRAM constraints.

How to run locally

Example using llama.cpp server:

bash
llama-server \
  -m mixdev-ya-q4_k_m.gguf \
  -ngl 999 \
  --host 0.0.0.0 --port 8080 \
  -c 32768