saadxsalman/Q-SS-0.5B-Reasoning-Math-GGUF
077
Q-SS-0.5B-Reasoning-Math-GGUF
The same structured math reasoning model — quantized and ready for instant local CPU inference.
Q-SS-0.5B-Reasoning-Math-GGUF is the quantized GGUF version of Q-SS-0.5B-Reasoning-Math, a fine-tuned Qwen/Qwen2.5-0.5B-Instruct trained with GRPO reinforcement learning on mathematical reasoning tasks. At just ~300MB with Q4KM quantization, it runs instantly on any CPU with no GPU required.
🔧 Want the full precision model for GPU or fine-tuning? See Q-SS-0.5B-Reasoning-Math.
✨ Highlights
- ⚡ Instant CPU inference — ~300MB Q4KM, runs on any machine
- 🧠 Thinks out loud — explicit step-by-step reasoning inside
<thought>tags - 🎯 Clean structured output — final answer always isolated in
<answer>tags - 🖥️ No GPU required — perfect for local, offline, and edge deployments
- 🔓 Apache 2.0 — free for personal and commercial use
📋 Model Details
💬 Output Format
Every response follows this strict structure:
<thought>
[Step-by-step reasoning and calculations]
</thought>
<answer>
[Final numerical answer only]
</answer>🚀 Quick Start
llama.cpp
# Download the model
huggingface-cli download saadxsalman/Q-SS-0.5B-Reasoning-Math-GGUF \\
--local-dir ./Q-SS-0.5B-Reasoning-Math-GGUF
# Run inference
./llama-cli \\
-m Q-SS-0.5B-Reasoning-Math-GGUF/model-q4_k_m.gguf \\
--temp 0.1 \\
-n 384 \\
-p "You are a mathematical reasoning engine. Solve the problem step-by-step inside <thought> tags, then give ONLY the final answer inside <answer> tags.\\n\\nProblem: Janet has 3 cats. Each cat eats 2 cans per day. How many cans for 7 days?"Ollama
ollama run hf.co/saadxsalman/Q-SS-0.5B-Reasoning-Math-GGUFPython with llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path = "./Q-SS-0.5B-Reasoning-Math-GGUF/model-q4_k_m.gguf",
n_ctx = 2048,
n_threads = 4,
)
SYSTEM_PROMPT = \"\"\"You are a mathematical reasoning engine.
Solve the problem step-by-step inside <thought> tags, then give ONLY the
final numerical or LaTeX result inside <answer> tags.
<thought>
[Your internal reasoning and calculations here]
</thought>
<answer>
[Final answer only]
</answer>\"\"\"
def solve(problem):
response = llm.create_chat_completion(
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": problem},
],
max_tokens = 384,
temperature = 0.1,
)
answer = response["choices"][0]["message"]["content"]
if "<answer>" in answer:
return answer.split("<answer>")[-1].split("</answer>")[0].strip()
return answer
print(solve("Janet has 3 cats. Each cat eats 2 cans of food per day. How many cans does she need for 7 days?"))
# Output: 42📝 Example Outputs
Problem: Janet has 3 cats. Each cat eats 2 cans of food per day. How many cans does she need for 7 days?
<thought>
Each cat eats 2 cans per day.
Janet has 3 cats, so they eat 3 × 2 = 6 cans per day together.
For 7 days: 6 × 7 = 42 cans total.
</thought>
<answer>
42
</answer>Problem: Tom has $50. He buys a book for $12 and a pen for $3. How much money does he have left?
<thought>
Tom starts with $50.
He spends $12 on a book and $3 on a pen.
Total spent: 12 + 3 = $15.
Money remaining: 50 - 15 = $35.
</thought>
<answer>
35
</answer>✅ What It's Good At
🖥️ Performance on CPU
Speeds are approximate and depend on system load and memory bandwidth.
📦 Related Models
⚠️ Limitations
- Optimized for English language math problems only
- Complex abstract reasoning, geometry, and calculus are beyond reliable capacity at 0.5B scale
- Q4KM quantization introduces minor precision loss vs full FP16 — negligible for most use cases
- Always verify critical calculations — the model may occasionally produce confident but incorrect answers
🙏 Acknowledgements
- Unsloth — efficient fine-tuning framework
- Qwen Team — Qwen2.5-0.5B-Instruct base model
- HuggingFace TRL — GRPO implementation
- llama.cpp — GGUF conversion and inference
- OpenR1 — OpenR1-Math-220k dataset
- OpenAI — GSM8K dataset
📄 Citation
@misc{qss-reasoning-math-gguf-2025,
author = {Saad Salman},
title = {Q-SS-0.5B-Reasoning-Math-GGUF},
year = {2025},
publisher = {HuggingFace},
howpublished = {\\url{https://huggingface.co/saadxsalman/Q-SS-0.5B-Reasoning-Math-GGUF}},
}