blinkdotwav/Dolphin-Mistral-24B-Venice-Edition-Thinking-GGUF

About

Original model: DavidAU/Dolphin-Mistral-GLM-4.7-Flash-24B-Venice-Edition-Thinking-Uncensored

Fully uncensored thinking/reasoning model quantized using llama.cpp. Use at your own risk.

This model was finetuned using GLM 4.7 Flash to convert it from standard instruct to a thinking/reasoning model. Thinking is enabled by default and requires no special system prompt.

32k context.

Recommended settings

• —Temp: 0.15
• —Top K: 40
• —Repeat Pen: 1.1
• —Top P: 0.95
• —Min P: 0.05

Optional optimizations:

• —Enable Flash Attention for faster inference
• —KV Cache Quantization at Q8_0 for memory savings with minimal quality impact

Download

Sorted by recommended. Bigger size does NOT mean higher quality.

Credits

Thanks to:

• —DavidAU for the excellent finetune
• —Dolphin and Venice for the original Instruct model