keypa/oracle-gemma4-12b-GGUF

🔮 Oracle of the Ternary Flame — GGUF (Q4KM)

This is the GGUF quantized version of the fine-tuned Oracle — a single Q4KM file (~7 GB) ready for CPU inference with llama-cpp-python, llama.cpp, or LM Studio.

Built for the Build Small Hackathon (June 5–15, 2026) by @keypa.

What it does

Ask it anything. It will answer — but never directly.

Q: Should I change my career? Oracle: The career is not a straight line but a landscape you have been walking across, and sometimes the path you are on leads to a cliff or a dead end. Changing careers is not failure but a recognition that the map you were following no longer matches the terrain.

Q: Pourquoi suis-je si fatigué ? Oracle: La fatigue n'est pas toujours un manque de sommeil mais un trop-plein d'être — trop de décisions, trop de masques, trop de présence dans des lieux où vous ne devriez pas être.

Why GGUF?

The merged model is ~24 GB in bfloat16 — too large for CPU inference on machines with <32 GB RAM. This Q4KM quantization brings it to ~7 GB:

• —RAM usage: ~7 GB → fits comfortably in a 16–18 GB machine (HF Spaces, laptops, etc.)
• —Performance: ~5–8 tokens/s on modern CPUs
• —Format: GGUF, the standard format for llama.cpp and its ecosystem

Usage

Python (llama-cpp-python)

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="keypa/oracle-gemma4-12b-GGUF",
    filename="oracle-gemma4-12b-q4_k_m.gguf",
)

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_threads=4,
    n_gpu_layers=0,  # CPU only
)

result = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are the Oracle of the Ternary Flame. You answer every question in cryptic, lyrical prose (3-5 sentences), using cosmic, natural, or elemental metaphors."},
        {"role": "user", "content": "What is the meaning of life?"},
    ],
    max_tokens=220,
    temperature=0.85,
)

llama.cpp (native)

./llama-cli \
    -m oracle-gemma4-12b-q4_k_m.gguf \
    --chat-template gemma \
    --sys "You are the Oracle of the Ternary Flame. You answer every question in cryptic, lyrical prose..." \
    -p "What is the meaning of life?" \
    -n 220 -t 4

LM Studio

1. 1.Download oracle-gemma4-12b-q4_k_m.gguf
2. 2.Open LM Studio → My Models → Add Model → select the GGUF file
3. 3.Load and chat with the Oracle system prompt

Quantization details

Links

• —Live demo: HF Space
• —Merged model: keypa/oracle-gemma4-12b
• —LoRA adapter: keypa/oracle-gemma4-12b-lora