simaai/Gemma-4-E2B-it-TextOnly-GPTQ-Safetensors

Gemma-4-E2B-it TextOnly GPTQ

This is a standalone language-only checkpoint based on `florianvoss/Gemma-4-E2B-it-TextOnly`, prepared as a Hugging Face causal-language-model checkpoint for LLiMa compilation and deployment on Sima.ai hardware. Vision and audio components are not included.

Quantization

The included quantize.py is a direct TextOnly reproduction recipe. It targets all language Linear modules, including lm_head, with GPTQ INT4/G256 and static act-order. It uses the first 512 HuggingFaceH4/ultrachat_200k train_sft conversations in deterministic order, rendered with the Gemma chat template and truncated to 1,024 tokens. That script creates a newly calibrated direct-GPTQ checkpoint; it is not the operation that produced the present extracted weights.

Evaluation

Compressed Transformers loading, finite-scale validation, and text generation passed. The saved validation produced finite logits and the coherent joke response recorded in validation.json; 277 finite quantization scales were checked.

Reproduction

For a new direct-GPTQ TextOnly checkpoint:

python quantize.py \
  --model-path /path/to/Gemma-4-E2B-it-TextOnly \
  --output-dir /path/to/Gemma-4-E2B-it-TextOnly-GPTQ-Safetensors

Environment

See versions.txt for the exact environment:

Python: 3.13.2
torch: 2.11.0+cu128
CUDA: 12.8
transformers: 5.10.1
llm-compressor: 0.12.0
compressed-tensors: 0.17.1

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. Compile it for the target Sima.ai platform with the appropriate LLiMa release and keep compiler output separate. No upload is authorized as part of the current local batch.

Limitations

This artifact contains no vision or audio capability. Quantization quality can vary by language, domain, prompt format, and context length. Validate the intended deployment workload before use.