constructai/Qwenite3.5-2B-GGUF

💥 Qwenite3.5-2B-GGUF

📄 Overview

Quant types

🎯 Intended Use

This model is designed for step‑by‑step reasoning tasks where the answer requires logical decomposition before the final response. It is optimized for:

• —Educational applications — explaining "why" and "how" questions
• —On‑device assistants — runs on mobile, Raspberry Pi, or CPU‑only environments in q4_k_m
• —Reasoning distillation research — studying how small models learn from large ones (Granite → Qwen)

Not recommended for: multimodal tasks, non‑reasoning chat (e.g., creative writing), or production systems requiring 100% factual accuracy.

⚠️ Limitations & Intended Use

Intended Use:

• —Educational & Reasoning tasks — explaining step‑by‑step logic (math, science, common sense)

• —On‑device assistants — runs on CPU, Raspberry Pi, mobile (small footprint, fast inference) in q4_k_m

• —Research baseline — for studying SFT‑only reasoning without RLHF/DPO

• —Distillation experiments — testing how well small models learn from large (Granite → Qwen)

Limitations:

• —Size matters — 2B parameters, so complex or multi‑hop reasoning may still fail

• —No multimodal — text only; images, video, audio are not supported

• —Factual accuracy — may hallucinate or give incorrect answers; always verify critical outputs

• —Domain restricted — trained on 15,000 reasoning examples (2 epochs); general chat or creative writing may be suboptimal

• —Training data bias — inherits biases from constructai/Granite-v4.1-Distilled-15K dataset; not safety‑filtered for harmful content

• —Hardware specific — optimised for T4/consumer GPUs; very slow on CPU without quantisation

🙏 Acknowledgements

This project would not have been possible without the open‑source community and the following resources:

• —Qwen Team (Alibaba Cloud) — for releasing the Qwen3.5-0.8B-Base model under Apache 2.0, a perfect balance of size and intelligence.

• —Unsloth AI — for making fine‑tuning on consumer hardware fast and memory‑efficient.

• —Hugging Face — for the ecosystem (transformers, datasets, PEFT, Hub) that democratises LLM training.

• —Kaggle — for providing free T4 GPU runtime to run this experiment.

📖 Citation

@misc{Qwenite3.5-2B-GGUF,
  author = {constructai},
  title = {Qwenite3.5-2B: Small Reasoning Model via SFT on Granite Traces},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {https://huggingface.co/constructai/Qwenite3.5-2B-GGUF},
}