Thox-ai/ThoxRoute-1.5B-GGUF

ThoxRoute-1.5B-GGUF

Your AI. Your Data. Your Rules.

Q4KM GGUF of ThoxRoute-1.5B (Qwen2.5-1.5B base) for on-device routing.

What this is

• —Merged + quantized from Thox-ai/ThoxRoute-1.5B → Qwen2.5-1.5B-Instruct (Apache-2.0).
• —Single Q4KM build (~0.99 GB). Infrastructure/private.

Intended use

On-device / edge text generation within the THOX stack. Not a safety-aligned public assistant unless deployed behind THOX guardrails.

Prompt / chat template

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Usage

llama.cpp

huggingface-cli download Thox-ai/ThoxRoute-1.5B-GGUF --include '*.gguf' --local-dir ./ThoxRoute-1.5B-GGUF
llama-cli -m ./ThoxRoute-1.5B-GGUF/model-Q4_K_M.gguf -p "Hello"

Links

• —Base model: `Thox-ai/ThoxRoute-1.5B`
• —Weights source (HF): `Thox-ai/ThoxRoute-1.5B`
• —Ollama: ollama.com/thox-ai/<slug> — verify with the Ollama lane (task 80017303)
• —Docs: https://docs.thox.ai

THOX.ai LLC — Your AI. Your Data. Your Rules. · On-device and private by design.