Modelpublic

simaai/Gemma-2-2B-it-Autoround-Safetensors

sourceHugging Facegemmaupdated 26d agoView on Hugging Face

0likes174downloads

Model Card

Gemma-2-2B-it Autoround

This is a post-training quantized version of `google/gemma-2-2b-it`, prepared as a Hugging Face checkpoint for LLiMa compilation and deployment on Sima.ai hardware.

Source revision: 299a8560bedf22ed1c72a8a11e7dce4a7f9f51f8

This checkpoint remains subject to the source model's license, intended-use guidance, and limitations.

Quantization

Component	Method	Weight format	Details
Decoder Linear layers	AutoRound	symmetric INT4, G256	182 targets; `lm_head` excluded; 200 iterations
`lm_head`	GPTQ	symmetric INT4, G256	static act-order; block size 128; dampening 0.01
Mixed-precision exceptions	Source precision	BF16	Embeddings, norms, and non-Linear operations

Calibration used the first 512 examples from HuggingFaceH4/ultrachat_200k train_sft, in deterministic order. Messages were rendered with the model chat template and packed into 512 complete 1,024-token spans. AutoRound batch size was 1. Calibration shuffling was disabled; the effective ordering is therefore deterministic.

Evaluation

Perplexity used the full wikitext-2-raw-v1 test split through the wikitext lm-eval task, without an example limit.

Checkpoint	Word perplexity	Status
Source checkpoint	17.152318	Full run: 2026-07-19
This UltraChat checkpoint	19.344819	Full run: 2026-07-19
Absolute degradation vs source	+2.192501	Lower is better
Relative degradation vs source	+12.78%	`100 * (quantized / source - 1)`

Raw JSON evidence is retained under perplexity_results/full_wikitext/.

Finite-scale validation and a Transformers chat-generation smoke test passed on 2026-07-19.

Reproduction

This directory includes the exact quantize.py, recipe.yaml, and versions.txt used for this artifact:

bash

python quantize.py \
  --model-path /path/to/models--google--gemma-2-2b-it \
  --output-dir /path/to/Gemma-2-2B-it-Autoround-Safetensors

Environment

See versions.txt for the exact environment:

text

Python: 3.13.2
torch: 2.11.0+cu128
CUDA: 12.8
transformers: 5.10.1
llm-compressor: 0.12.0
auto-round: 0.13.0
compressed-tensors: 0.17.1

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. Compile it with the appropriate LLiMa release and keep compiler output separate from this checkpoint. No upload is authorized as part of the current local batch.

Limitations

Quantization can affect quality, especially for tasks unlike the calibration data. Validate this checkpoint for the intended language, domain, prompt format, context length, safety requirements, and deployment runtime.