Modelpublic

darioooooo0o/K2-Horizon-7B-GGUF

sourceHugging Faceapache-2.0updated 23d agoView on Hugging Face

0likes750downloads

Model Card

K2-Horizon-7B GGUF quants

![X](https://x.com/imdariotoo)

Requests, questions or suggestions? Message me on X: https://x.com/imdariotoo

GGUF quantizations of IFM/K2-Horizon-7B.

Converted with the official k2-official llama.cpp branch (MBZUAI-IFM port, commit 35999d101).

Files

File	Quant	Size
`k2horizon7-q3_k_m.gguf`	Q3KM	~4.3 GB
`k2horizon7-q4_k_s.gguf`	Q4KS	~5.0 GB
`k2horizon7-q4_k_m.gguf`	Q4KM	~5.2 GB
`k2horizon7-q5_k_m.gguf`	Q5KM	~6.0 GB
`k2horizon7-q8_0.gguf`	Q8_0	~8.9 GB

Requirements

Use a llama.cpp build from the `k2-official` branch of MBZUAI-IFM/llama.cpp (or anything that merges that port). Mainline llama.cpp does NOT support the k2-horizon architecture.

Usage

bash

llama-cli -m k2horizon7-q4_k_m.gguf -ngl 99 -c 8192

Fits entirely on an 8 GB+ GPU; no CPU offload needed.

Notes

—Plain K-quants from BF16, no imatrix.
—All quants verified loading and generating on RTX 3060 12GB.