Modelpublic

positron-ai/meta-llama_Llama-3.2-1B-Instruct-tron-best-gptq-permuted

sourceHugging Facellama3.2updated 28d agoView on Hugging Face

0likes38downloads

Model Card

Positron AI Quantized Build

Built with Llama.

This repository contains a Positron AI quantized build of meta-llama/Llama-3.2-1B-Instruct for inference.

Recommended Use

Use this artifact when you need a GPTQ 4-bit build of meta-llama/Llama-3.2-1B-Instruct built by Positron AI.

For general-purpose GPU inference, compare against the original model and other quantized formats before deployment.

Artifact Summary

Field	Value
Base model	meta-llama/Llama-3.2-1B-Instruct
Published artifact	positron-ai/meta-llama_Llama-3.2-1B-Instruct-tron-best-gptq-permuted
Quantization method	GPTQ
Quantization format	gptq
Source precision	n/a
Target runtime	n/a
Hardware target	n/a
Release date	2026-08-25
License	llama3.2 (Llama 3.2 Community License)

Quantization Details

Field	Value
Weight precision	4-bit
Activation precision	not quantized
Bits	4
Group size	64
Symmetric quantization	true
Activation ordering / desc_act	true
Damp percent	0.05
Calibration dataset	Mixed-domain calibration set
Calibration samples	256
Calibration sequence length	2048
MoE experts per token	n/a
Quantization toolchain	GPTQModel 5.8.0, transformers 4.57.6, torch 2.9.1, CUDA 12.8

License

This model is a derivative of Llama 3.2 and is distributed under the Llama 3.2 Community License (LICENSE.txt). The repository also includes Meta's Acceptable Use Policy (USE_POLICY.md) and the attribution notice required by the license (NOTICE).

Provenance

This artifact was produced by Positron AI from meta-llama/Llama-3.2-1B-Instruct via GPTQ quantization using GPTQModel. The original model license and usage restrictions continue to apply.