Modelpublic

glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF

sourceHugging Faceapache-2.0updated 9mo agoView on Hugging Face

1likes368downloads

Model Card

granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF

GGUF quantized versions of granite-4.0-h-tiny-DISTILL-OPUS-4.5-think

Available Formats

Filename	Size	Quant Type
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-f16.gguf	13.22 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-F16
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q2_k.gguf	2.46 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q2_K
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q3_k_l.gguf	3.42 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q3KL
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q3_k_m.gguf	3.18 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q3KM
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q3_k_s.gguf	2.91 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q3KS
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q4_0.gguf	3.77 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q4_0
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q4_1.gguf	4.18 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q4_1
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q4_k_m.gguf	4.02 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q4KM
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q4_k_s.gguf	3.80 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q4KS
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q5_0.gguf	4.58 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q5_0
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q5_1.gguf	4.98 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q5_1
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q5_k_m.gguf	4.71 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q5KM
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q5_k_s.gguf	4.58 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q5KS
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q6_k.gguf	5.44 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q6_K
granite-4.0-h-tiny-DISTILL-4.5-opus-high-think-q8_0.gguf	7.04 GB	GRANITE-4.0-H-TINY-DISTILL-4.5-OPUS-HIGH-THINK-Q8_0

Quick Start

Ollama

bash

# Use Q4_K_M (recommended)
ollama run hf.co/glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF:Q4_K_M

# Or other quantizations
ollama run hf.co/glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF:Q8_0
ollama run hf.co/glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF:Q2_K

llama.cpp

bash

# Download and run
llama-cli --hf-repo glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF --hf-file granite-4.0-h-tiny-distill-opus-4.5-think-q4_k_m.gguf -p "Hello, how are you?"

# With server
llama-server --hf-repo glogwa68/granite-4.0-h-tiny-DISTILL-OPUS-4.5-think-GGUF --hf-file granite-4.0-h-tiny-distill-opus-4.5-think-q4_k_m.gguf -c 2048

LM Studio / GPT4All

Download the .gguf file of your choice and load it in your application.

Quantization Details

Type	Bits	Use Case
Q2_K	2	Extreme compression, low quality
Q3KM	3	Very compressed
Q4KM	4	Recommended - Best size/quality
Q5KM	5	High quality
Q6_K	6	Very high quality
Q8_0	8	Near lossless
F16	16	Original precision

Original Model

This is the quantized version of granite-4.0-h-tiny-DISTILL-OPUS-4.5-think

—Base Model: ibm-granite/granite-4.0-h-tiny
—Fine-tuning Dataset: TeichAI/claude-4.5-opus-high-reasoning-250x
—Special Feature: Thinking/Reasoning with <think> tags