Modelpublic

darioooooo0o/K2-Horizon-3.7B-GGUF

sourceHugging Faceapache-2.0updated 23d agoView on Hugging Face

1likes561downloads

Model Card

K2-Horizon-3.7B GGUF quants

![X](https://x.com/imdariotoo)

Requests, questions or suggestions? Message me on X: https://x.com/imdariotoo

GGUF quantizations of IFM/K2-Horizon-3.7B.

Converted with the official k2-official llama.cpp branch (MBZUAI-IFM port, commit 35999d101).

Files

File	Quant	Size
`k2horizon37-q3_k_m.gguf`	Q3KM	~2.5 GB
`k2horizon37-q4_k_s.gguf`	Q4KS	~2.8 GB
`k2horizon37-q4_k_m.gguf`	Q4KM	~2.9 GB
`k2horizon37-q5_k_m.gguf`	Q5KM	~3.4 GB
`k2horizon37-q8_0.gguf`	Q8_0	~5.0 GB

Requirements

Use a llama.cpp build from the `k2-official` branch of MBZUAI-IFM/llama.cpp (or anything that merges that port). Mainline llama.cpp does NOT support the k2-horizon architecture.

Usage

bash

llama-cli -m k2horizon37-q4_k_m.gguf -ngl 99 -c 8192

Fits entirely on any modern GPU; no CPU offload needed.

Notes

—Plain K-quants from BF16, no imatrix.
—All quants verified loading and generating on RTX 3060 12GB.