caiovicentino1/Harmonic-27B-HLWQ-Q5

[!IMPORTANT] Naming notice (2026-04-10). The "HLWQ" technique used in this model is being rebranded to HLWQ (Hadamard-Lloyd Weight Quantization). The change is only the name; the algorithm and the weights in this repository are unchanged. The rebrand resolves a name collision with an unrelated, earlier KV cache quantization method also named HLWQ (Han et al., arXiv:2502.02617, 2025). HLWQ addresses weight quantization with a deterministic Walsh-Hadamard rotation and Lloyd-Max scalar codebook; Han et al.'s HLWQ addresses KV cache quantization with a random polar rotation. The two methods are technically distinct. Existing loaders that load this repository by ID continue to work without changes. Future model uploads will use the HLWQ name. Reference paper for this technique: arXiv:2603.29078 (v2 in preparation; v1 still uses the old name).

Harmonic-27B — HLWQ INT4

Native vLLM. Marlin kernel. Zero plugin.

HLWQ Q5 preprocessing produces better INT4 weights than direct quantization — stored in CompressedTensors format for native vLLM inference.

Quick Start — vLLM (one command)

pip install vllm
vllm serve caiovicentino1/Harmonic-27B-HLWQ-Q5 --language-model-only --enforce-eager

That's it. No plugin, no pip install polarquant, no custom code.

Tested results:

Quick Start — HuggingFace Transformers

pip install polarquant

import polarengine_vllm  # auto-registers with transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("caiovicentino1/Harmonic-27B-HLWQ-Q5", device_map="auto", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("caiovicentino1/Harmonic-27B-HLWQ-Q5", trust_remote_code=True)

inputs = tokenizer("Hello!", return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Consumer GPU Compatibility

Why HLWQ INT4 is Better

Standard INT4 (GPTQ/AWQ) quantizes weights directly — outliers cause errors.

HLWQ adds a preprocessing step:

1. 1.Hadamard rotation — distributes weight energy uniformly (eliminates outliers)
2. 2.Lloyd-Max Q5 — MSE-optimal quantization for the resulting Gaussian distribution
3. 3.Dequant → INT4 — the cleaned weights produce better INT4 than direct quantization

Same speed as GPTQ/AWQ, better quality.

Important Flags

Links

• —Paper: arxiv.org/abs/2603.29078
• —GitHub: github.com/caiovicentino/polarengine-vllm
• —PyPI: pip install polarquant
• —Base model: Qwen/Qwen3.5-27B