Modelpublic

EchoLabs33/smollm3-3b-hxq

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes19downloads

Model Card

SmolLM3-3B-HXQ (GGUF)

Native GGUF HXQ_AFFINE_6 quantization of SmolLM3-3B for llama.cpp. 6.28 bits per weight, calibration-free affine quantization. Runs at Q8_0 parity speed (28.3 vs 28.4 tok/s) at 26% smaller size.

Quick Start

bash

# With llama.cpp (HXQ fork)
./llama-cli -m smollm3-3b-hxq-affine6.gguf -p "Explain quicksort:" -n 128

Benchmarks (Quadro T2000 4GB)

Quant	BPW	Size	PPL (WikiText-2)	vs Q8_0	tg128 tok/s
Q8_0	8.50	3.04 GiB	9.399	baseline	28.4
HXQ_AFFINE_6	6.28	2.25 GiB	9.520	+1.28%	28.3
Q4KM	4.96	1.78 GiB	9.656	+2.72%	44.0

Details

—Source: HuggingFaceTB/SmolLM3-3B (safetensors → F16 GGUF → HXQAFFINE6)
—Quantization: Single-pass F16 → HXQ (no double-quantization)
—Compatibility: Requires llama.cpp HXQ fork (hxq-affine-type branch)
—Architecture: SmolLM3 (transformer, 3B parameters)

About HXQ

HXQ is a calibration-free vector quantization method for neural network weights. It uses per-group-128 affine coding to achieve 6.28 bits per weight with minimal perplexity degradation. See HXQ whitepaper for details.

Built by EchoLabs33.