CoolFace
Modelpublic

1bit-MONSTER/ZAYA1-8B-GGUF

sourceHugging Faceapache-2.0updated 4h agoView on Hugging Face
0likes
Model Card

ZAYA1-8B — GGUF

Our own GGUF conversion of Zyphra's ZAYA1-8B, ported and validated as part of the 1bit engine's architecture showcase. This is a from-scratch conversion (our own zaya.py in the engine's llama.cpp fork), not a re-host of an existing GGUF — ZAYA1 wasn't previously available in GGUF form.

Contents

  • —zaya1-8b-f16.gguf — full-precision conversion, for further requantization.
  • —zaya1-8b-Q4_K_M.gguf — the quant we benchmark and recommend for serving.

Validation

Checked against the original transformers model in FP32 on the CPU, teacher-forced top-1 agreement: 95/96 positions for F16. For scale, the HF BF16 checkpoint itself agrees at 91/96 against FP32, so the conversion loses less than BF16 rounding does. Tokenization matches the model's own tokenizer exactly (a wikitext passage, code, and mixed-script text).

Measured performance (Strix Halo, Q4KM)

BackendDecode
Vulkan93 tok/s (pp512: 3437 tok/s)
ROCm61.5 tok/s
HRX25.5 tok/s

Vulkan is the clear pick for this model on this hardware.

Running it

With the 1bit engine:

bash
1bit serve -m zaya1-8b-Q4_K_M.gguf --device vulkan

ZAYA1 support (the zaya GGUF architecture, its conversion path, and the Vulkan/ROCm/HRX kernels) lives on the engine's 1bit-MONSTER/llama.cpp fork — a plain upstream llama.cpp checkout will not load this GGUF's tensor layout (cca_conv_grp is tap-major, rope theta 5e6 from the model's hybrid rope parameters).

Attribution

  • —Base model: Zyphra/ZAYA1-8B, Apache 2.0.
  • —GGUF conversion, kernel ports, and validation: this project's engine team.
  • —License: Apache 2.0, inherited from the base model.