Modelpublic

build-small-hackathon/deku-gguf

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes20downloads

Model Card

Deku — GGUF (llama.cpp)

GGUF builds of **build-small-hackathon/deku**, the One for All student: a Qwen2.5-0.5B distilled from 6 teachers via gated CKA geometry distillation. The LoRA adapter is merged into the base, then converted with llama.cpp's convert_hf_to_gguf.py.

Files

File	Size	Use
`deku-q8_0.gguf`	~531 MB	what the Space serves — near-lossless, CPU-friendly
`deku-f16.gguf`	~994 MB	archival full-precision build
`gating.npz`	~22 KB	the teacher-gating head as numpy (`weight` 6×896, `bias` 6)

Run

bash

llama-cli -m deku-q8_0.gguf -p "Explain gradient descent in one sentence."

python

from llama_cpp import Llama
llm = Llama(model_path="deku-q8_0.gguf", n_ctx=2048)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Why is the sky blue?"}]
)["choices"][0]["message"]["content"])

Teacher gating without torch

gating.npz lets you reproduce the live "teacher influence" meters from the Space using only numpy on a mean-pooled embedding from llama.cpp:

python

import numpy as np
g = np.load("gating.npz")          # g["weight"] (6, 896), g["bias"] (6,)
def gate(emb):                     # emb: 896-dim pooled embedding
    z = g["weight"] @ emb + g["bias"]
    e = np.exp(z - z.max())
    return e / e.sum()             # softmax over the 6 teachers

Teacher order: qwen, smollm, phi, gemma, minicpm, nemotron.