Modelpublic

simaai/LFM2.5-1.2B-Instruct-Autoround-Safetensors

sourceHugging Facemitupdated 26d agoView on Hugging Face

0likes199downloads

Model Card

LFM2.5-1.2B-Instruct Autoround

Pre-LLiMa Hugging Face checkpoint based on `LiquidAI/LFM2.5-1.2B-Instruct` at source revision 6314d2b7cf28a6ae9de9d3e77dcfcd9c9f281c77. It remains subject to the base model's license, intended use, and limitations.

Quantization

Component	Method	Weight format	Details
Decoder Linear layers	AutoRound	symmetric INT4, G256	92 targets; `lm_head` excluded
`lm_head`	GPTQ	symmetric INT4, G256	static act-order, block size 128, dampening 0.01
Mixed-precision exceptions	—	source dtype	non-Linear parameters remain at source precision

Calibration used HuggingFaceH4/ultrachat_200k, train_sft[:512], rendered with the source chat template. Token IDs were deterministically concatenated and packed into 512 full 1024-token spans (no shuffle); AutoRound used 200 iterations and batch size 1.

Evaluation

Full wikitext-2-raw-v1 evaluation used the wikitext lm-eval task, no example limit, batch size 1, CUDA, and the same evaluator for source and quantized checkpoints on 2026-07-19.

Checkpoint	Word perplexity	Status
`LiquidAI/LFM2.5-1.2B-Instruct` source	33.871271	Full run
This UltraChat checkpoint	42.956951	Full run
Absolute degradation	9.085681	Lower is better
Relative degradation	26.824%	`100 * (quantized / source - 1)`

UltraChat is the selected calibration corpus because the intended deployment prioritizes instruction-following data. Finite-scale validation and a Transformers chat-generation smoke test passed.

Reproduction

This directory includes the exact quantize.py, recipe.yaml, and versions.txt.

bash

python quantize.py --model-path /path/to/models--LiquidAI--LFM2.5-1.2B-Instruct --output-dir /path/to/output

Environment

Exact Python, CUDA, Torch, Transformers, llm-compressor, AutoRound, and compressed-tensors versions are recorded in versions.txt.

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for the target Sima.ai platform and keep compiler output separate. No upload is authorized as part of the current local batch.

Limitations

WikiText perplexity does not directly measure instruction following. Quantization quality can vary by language, domain, prompt format, context length, safety requirements, and deployment runtime; validate the intended workload independently.