CoolFace
Modelpublic

prithivMLmods/Qwen3-4B-Esper3-F32-GGUF

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes178downloads
Model Card

Qwen3-4B-Esper3-F32-GGUF

Esper 3 is a specialist model built on Qwen 3, designed for coding, architecture, and DevOps reasoning. It has been fine-tuned using our proprietary DevOps, architecture, and code reasoning dataset generated with Deepseek R1. This tuning enhances its general and creative reasoning abilities, making it effective not only in problem-solving but also in general conversation. With its small model sizes, Esper 3 is optimized for fast inference, making it suitable for deployment on local desktops, mobile devices, and high-speed server environments.

Model Files

FilenameSizeFormatDescription
Qwen3-4B-Esper3.BF16.gguf8.05 GBBF16Brain Float 16-bit quantization
Qwen3-4B-Esper3.F16.gguf8.05 GBF16Half precision (16-bit) floating point
Qwen3-4B-Esper3.F32.gguf16.1 GBF32Full precision (32-bit) floating point
Qwen3-4B-Esper3.Q2_K.gguf1.67 GBQ2_K2-bit quantization with K-quant
Qwen3-4B-Esper3.Q3KL.gguf2.24 GBQ3KL3-bit quantization (Large) with K-quant
Qwen3-4B-Esper3.Q3KM.gguf2.08 GBQ3KM3-bit quantization (Medium) with K-quant
Qwen3-4B-Esper3.Q3KS.gguf1.89 GBQ3KS3-bit quantization (Small) with K-quant
Qwen3-4B-Esper3.Q4KM.gguf2.5 GBQ4KM4-bit quantization (Medium) with K-quant
Qwen3-4B-Esper3.Q4KS.gguf2.38 GBQ4KS4-bit quantization (Small) with K-quant
Qwen3-4B-Esper3.Q5KM.gguf2.89 GBQ5KM5-bit quantization (Medium) with K-quant
Qwen3-4B-Esper3.Q5KS.gguf2.82 GBQ5KS5-bit quantization (Small) with K-quant
Qwen3-4B-Esper3.Q6_K.gguf3.31 GBQ6_K6-bit quantization with K-quant
Qwen3-4B-Esper3.Q8_0.gguf4.28 GBQ8_08-bit quantization

Recommended Usage

  • —Q4_K_M or Q5_K_M: Best balance of quality and performance for most users
  • —Q6_K or Q8_0: Higher quality, larger file sizes
  • —Q2_K or Q3_K_S: Fastest inference, lower quality
  • —F16 or BF16: High quality, requires more VRAM
  • —F32: Highest quality, requires significant VRAM

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

image.png