VibeManGeo/Zen-5-Coder-GGUF

Zen-5-Coder GGUF

GGUF quantizations of Zen-5-Coder 80B for llama.cpp and compatible runtimes.

The original model was released by Zen LM in Hugging Face Transformers format. This repository provides converted and quantized GGUF versions optimized for local inference across a wide range of hardware configurations.

Overview

Available Quantizations

Conversion Pipeline

All files were generated locally using the standard llama.cpp workflow:

Hugging Face Transformers
        ↓
GGUF FP16
        ↓
GGUF Quantization

Tools Used

• —llama.cpp
• —converthfto_gguf.py
• —llama-quantize

Example Usage

llama.cpp

llama-cli \
  -m Zen-5-Coder-Q4_K_M.gguf \
  -c 32768 \
  -ngl 999 \
  -p "Write a Python web server"

llama-server

llama-server \
  -m Zen-5-Coder-Q4_K_M.gguf \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080

Hardware Used For Conversion

The quantizations in this repository were generated and tested on:

• —GPU 0 NVIDIA RTX 3060 12 GB Headless
• —GPU 1 NVIDIA Tesla P40 24 GB Headless
• —AMD Ryzen 7 5700G
• —64 GB DDR-4 3200Mhz System RAM
• —Debian Linux 13.2

Actual performance will depend on context size, quantization level, GPU offloading, and runtime configuration.

Credits

Original Model

Zen LM — creators of Zen-5-Coder.

GGUF Conversion & Quantization

VibeManGeo

Fun fact: these 80B quantizations were produced before the author passed CompTIA A+ Core 1.

Acknowledgements

Special thanks to the llama.cpp developers for providing the tools that make efficient local inference and GGUF quantization possible.

Disclaimer

This repository contains converted and quantized derivatives of the original model.

All credit for model architecture, training, datasets, and original weights belongs to the original authors.

Support the Original Authors

If these GGUF files save you the time and compute resources required for conversion and quantization, please consider supporting the original creators by visiting the original Zen-5-Coder model page.

Notes

These GGUF files were independently converted and quantized from the original Hugging Face release using llama.cpp.

The goal of this repository is to make Zen-5-Coder immediately accessible to the local inference community without requiring users to perform the conversion process themselves.