HattoriHanzo1/Qapricorn-4B-GGUF

Qapricorn-4B 🐐⚔️

"Nusquam est qui ubique est" — Kto jest wszędzie, nie jest nigdzie. Qapricorn jest tam gdzie trzeba — reasoning, nie encyklopedia.

Model Description

Qapricorn-4B is a fine-tuned version of Qwen3-4B with enhanced reasoning capabilities, Polish language support, and coding skills. The model was trained using QLoRA with a carefully curated multilingual dataset combining mathematical reasoning, coding tasks, and Polish language data.

The name Qapricorn comes from:

• —Qa — Qapla' (Klingon for "Success")
• —pricorn — Capricorn (perseverance, patience)

A model built with persistence, on a free GPU, step by step. 💪

Key Features

• —🧠 Native `<think>` reasoning — inherited from Qwen3-4B and strengthened through training
• —🇵🇱 Polish language support — thinks and responds in Polish when prompted in Polish
• —💻 Coding capability — Python, algorithms, data structures
• —🔢 Mathematical reasoning — step-by-step problem solving
• —🌍 Multilingual — seamlessly switches between Polish and English based on prompt language

Reasoning Example

Prompt: Stała Kaprekara to liczba 6174. Zastosuj algorytm dla liczby 1234.

<think>
Algorytm Kaprekara działa tak: 
- Iteracja 1: 4321 - 1234 = 3087
- Iteracja 2: 8730 - 0378 = 8352  
- Iteracja 3: 8532 - 2358 = 6174 ✓

Warto też wspomnieć że dla liczb trzycyfrowych analogiczna stała to 495...
</think>

Algorytm Kaprekara dla 1234:
- Iteracja 1: 4321 - 1234 = 3087
- Iteracja 2: 8730 - 0378 = 8352
- Iteracja 3: 8532 - 2358 = **6174** ✓

The model correctly solved the Kaprekar constant problem AND independently noted the analogous constant (495) for 3-digit numbers — demonstrating genuine reasoning, not pattern matching.

Training Details

Base Model

• —Model: unsloth/Qwen3-4B-unsloth-bnb-4bit
• —Method: QLoRA (4-bit quantization)
• —LoRA rank: r=16, alpha=32
• —Target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
• —Trainable parameters: 33M / 4B (0.81%)

Training Infrastructure

• —Platform: Kaggle (free tier) — 1x Tesla T4 16GB
• —Framework: Unsloth 2026.3.3 + TRL
• —Total steps: 1500
• —Dataset coverage: ~33% of training data

Training Phases

Loss Progression

Phase 1: ~1.8 → 1.1  (rapid learning)
Phase 2: avg 0.755    (best convergence)
Phase 3: avg 1.114    (new cosine cycle)
Phase 4: avg 1.076    (gradual improvement)
Phase 5: avg 1.046    (final polish, min=0.600)

Dataset

Mixed multilingual dataset (~48k records, 269MB):

All datasets were converted to ChatML format with <think> blocks preserved.

Available Formats

Usage

llama.cpp

./llama-cli \
  -m qapricorn_Q4_K_M.gguf \
  -p "Czym jest ciąg geometryczny? Wyjaśnij krok po kroku." \
  --temp 0.6 \
  --top-p 0.95 \
  -n 512

llama-server

./llama-server \
  -m qapricorn_Q4_K_M.gguf \
  --port 8080 \
  --ctx-size 4096 \
  --temp 0.6

Python (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("HattoriHanzo1/Qapricorn-4B-merged-bf16")
tokenizer = AutoTokenizer.from_pretrained("HattoriHanzo1/Qapricorn-4B-merged-bf16")

messages = [
    {"role": "system", "content": "Jesteś pomocnym asystentem AI."},
    {"role": "user", "content": "Wyjaśnij algorytm Kaprekara."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.6, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Thinking Mode

Qapricorn inherits Qwen3's thinking toggle:

/think    — enable thinking (default)
/no_think — disable thinking for faster responses

Benchmarks (Qualitative)

Limitations

• —Biology/natural science responses may contain occasional hallucinations (dataset gap)
• —4B parameter limit — complex multi-step reasoning may be less reliable than larger models
• —Training covered ~33% of available dataset — further training possible

Related Repositories

• —🔧 Qapricorn-4B-merged-bf16 — Full merged model
• —🎯 Qapricorn-4B-adapter-1500 — LoRA adapter (1500 steps)
• —💾 Qapricorn-4B-adapter-1100 — LoRA adapter (1100 steps)

License

Apache 2.0 — same as base model Qwen3-4B.

Built on a free GPU, on a Sunday, step by step. "Heghlu'meH QaQ jajvam" — Today is a good day to train. ⚔️