CoolFace
Modelpublic

rodrigomt/Qwen3-30B-A3B-Thinking-Deepseek-Distill-2507-v3.1-V2-GGUF

sourceHugging Faceupdated 1y agoView on Hugging Face
7likes337downloads
Model Card

🧠 Qwen3-30B-A3B-Thinking-2507-Deepseek-v3.1-Distill-V2 GGUFs

Quantized version of: BasedBase/Qwen3-30B-A3B-Thinking-2507-Deepseek-v3.1-Distill-V2-FP32

📦 Available GGUFs

FormatDescription
F16Full precision (16-bit), better quality, larger size ⚖️
Q5_K_XLQuantized (5-bit XL variant, based on the quantization table of the unsloth model Qwen3-30B-A3B-Thinking-2507), medium size, faster inference ⚡
Q4_K_XLQuantized (4-bit XL variant, based on the quantization table of the unsloth model Qwen3-30B-A3B-Thinking-2507), smaller size, faster inference ⚡
Q3_K_XLQuantized (3-bit XL variant, based on the quantization table of the unsloth model Qwen3-30B-A3B-Thinking-2507), smaller size, faster inference ⚡

🚀 Usage

Example with llama.cpp:

bash
./main -m ./gguf-file-name.gguf -p "Hello world!"