CoolFace
Modelpublic

NightPrince/Muslim-6B-PRO-GGUF

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
1likes624downloads
Model Card

<p align="center"> <img src="https://huggingface.co/NightPrince/Muslim-6B-PRO-GGUF/resolve/main/muslim-6b-pro-banner-light.png" alt="Muslim-6B-PRO" width="100%" /> </p>

Muslim-6B-PRO — GGUF

GGUF quantizations of NightPrince/Muslim-6B-PRO for local inference with `llama.cpp` and compatible runtimes (LM Studio, Ollama, koboldcpp, etc.).

Files

FileQuantSizeNotes
Muslim-6B-PRO-Q2_K.ggufQ2_K2.26 GBSmallest, largest quality loss
Muslim-6B-PRO-Q3_K_S.ggufQ3KS2.57 GB
Muslim-6B-PRO-Q3_K_M.ggufQ3KM2.83 GB
Muslim-6B-PRO-Q3_K_L.ggufQ3KL3.06 GB
Muslim-6B-PRO-Q4_0.ggufQ4_03.24 GBLegacy 4-bit
Muslim-6B-PRO-Q4_1.ggufQ4_13.56 GBLegacy 4-bit
Muslim-6B-PRO-Q4_K_S.ggufQ4KS3.26 GB
Muslim-6B-PRO-Q4_K_M.ggufQ4KM3.41 GBRecommended default — best size/quality balance
Muslim-6B-PRO-Q5_0.ggufQ5_03.88 GBLegacy 5-bit
Muslim-6B-PRO-Q5_1.ggufQ5_14.19 GBLegacy 5-bit
Muslim-6B-PRO-Q5_K_S.ggufQ5KS3.88 GB
Muslim-6B-PRO-Q5_K_M.ggufQ5KM3.96 GBNear-lossless, good balance for more headroom
Muslim-6B-PRO-Q6_K.ggufQ6_K4.55 GBVery close to F16 quality
Muslim-6B-PRO-Q8_0.ggufQ8_05.89 GBNear-lossless
Muslim-6B-PRO-f16.ggufF1611.08 GBFull precision, no quantization loss

Usage

llama.cpp

bash
llama-cli -hf NightPrince/Muslim-6B-PRO-GGUF:Q4_K_M -p "السلام عليكم"

or, with a locally downloaded file:

bash
llama-cli -m Muslim-6B-PRO-Q4_K_M.gguf -p "السلام عليكم"

Server mode (OpenAI-compatible API)

bash
llama-server -m Muslim-6B-PRO-Q4_K_M.gguf --port 8080

Tool calling

This model uses the Hermes-style <tool_call> format. llama-server supports this natively via its --jinja flag (uses the model's built-in chat template) combined with the standard OpenAI-style tools parameter in requests.

Choosing a quant

  • —Q4_K_M is the recommended default for most use — the standard "good enough for almost everyone" tradeoff.
  • —Go Q5_K_M or Q6_K if you have the VRAM/RAM headroom and want output closer to the original fp16 model, especially for tool-call argument precision.
  • —Go Q2_K/Q3_K only under tight memory constraints — expect noticeably more degradation on precise tasks like tool-call JSON formatting and exact surah/ayah references.
  • —Q8_0 or F16 for maximum fidelity when memory isn't a constraint.

Related resources