CogHyn/Llama-3.2-1B-Instruct-HadamardSpin-4bits

Llama-3.2-1B-Instruct-HadamardSpin-4bits

Github repo: github

🚀 Overview

HadamardSpin is a high-performance quantization project focusing on preserving the "structural intelligence" of Large Language Models (LLMs) at low-bit precision. This specific version features Llama-3.2-1B-Instruct quantized to 4-bit using a Hadamard-Transform-based Rotation technique, optimized for Apple Silicon via the MLX framework.

The core innovation lies in the Incoherence Processing: using a Hadamard matrix ($n=2048$) to scatter "super-outliers" across hidden dimensions, effectively crushing quantization noise that typically destroys logic in small models.

📈 Experimental Results

Knowledge & Reasoning (CommonSense)

We evaluated the model across ARC, Hellaswag (HS), and Winogrande (WG).

Logic & Coding (HumanEval)

Note: Despite a slightly higher Perplexity ($9.604$), the Hadamard-rotated model preserves "structural knowledge nodes," leading to superior reasoning performance over lower-PPL counterparts.

Performance on Apple M1

Measurements taken on a standard Apple M1 (8-core GPU, 8GB Unified Memory).

🛠️ How it Works: The "Incoherence" Strategy

Standard quantization (RTN) fails on Llama-3.2 because of extreme outliers (activations peaking at $\pm 400$). Mark-3 intervenes by:

1. 1.Flattening the Weights: Compressing weight ranges from $[-0.6, 0.6]$ down to $[-0.1, 0.1]$.
2. 2.Scattering Activations: Rotating the hidden space so no single dimension holds dominant energy, reducing peak activation amplitude by 10x.
3. 3.Hadamard-2048: Larger block sizes allow for better scattering ($1/\sqrt{n}$), making simple RTN quantization behave like an advanced optimizer.

Usage (MLX)

import mlx.core as mx
from mlx.lm_utils import load

model, tokenizer = load("CogHyn/Llama-3.2-1B-Instruct-HadamardSpin-4bits")

# Generate
response = model.generate("Explain the Markov Property in one sentence.")
print(response)

📜 Citation & Credits

Developed by Nguyễn Công Huynh as part of the "Mark-3" Graduation Thesis.

@thesis{huynh2026mark3,
  title={Mark-3: Advanced LLM Quantization via Orthogonal Rotation},
  author={Nguyen Cong Huynh},
  year={2026},
  institution={University of Engineering and Tecnology - VNU}
}