Modelpublic

simaai/Gemma-4-E4B-it-GPTQ-Safetensors

sourceHugging Faceapache-2.0updated 26d agoView on Hugging Face

0likes38downloads

Model Card

Gemma-4-E4B-it GPTQ

This is a local post-training quantized checkpoint of google/gemma-4-E4B-it for later LLiMa compilation and deployment on Sima.ai hardware. It must not be uploaded as part of the current VLM batch.

Source revision: Not captured in the flattened local source directory. An immutable upstream revision must be recovered and recorded before any release or redistribution.

This checkpoint remains subject to the upstream Apache-2.0 license, intended-use guidance, and limitations.

Quantization

Component	Method	Weight format	Effective targets
Decoder Linear layers and `lm_head`	GPTQ	symmetric INT4, group size 256, static act-order	344
Vision encoder Linear layers	RTN	symmetric INT8, per-channel	113
Audio tower	none	source BF16	134 explicit exceptions
Vision/audio embedding projectors	none	source BF16	2 explicit exceptions

The lm_head follows the default GPTQ INT4/G256 policy; no INT8 escalation was made. Text GPTQ calibration used lmms-lab/flickr30k test[:512], deterministic dataset order, 512 image-text samples, sequence length 2048, batch size 1, no concatenation, and no padding to maximum length. Vision RTN does not use Hessian calibration. Exact targets and exceptions are saved in recipe.yaml.

Evaluation

Full MMStar used all 1,500 examples, VLMEvalKit commit 7055d3010c38ccb5dcae1bc9535ca19c7fe5d79f, deterministic generation, a matched letter-only system prompt, maximum 16 new tokens, SDPA, and exact_matching without an API judge.

Checkpoint	Overall accuracy
Source	47.1333%
Existing text GPTQ + vision GPTQ checkpoint	40.4667%
This text GPTQ + vision RTN checkpoint	43.4667%
RTN change versus vision GPTQ	+3.0000 percentage points
RTN relative change versus vision GPTQ	+7.4135%

Evaluation date: 2026-07-17. Slurm jobs: source 4708, vision-GPTQ baseline 4709, and this vision-RTN candidate 4742. The comparison is recorded in vlmeval_results/mmstar_results_vision_rtn.csv; raw predictions and status files remain under vlmeval_results/mmstar_full_vision_rtn/.

Validation: MMStar inference passed (1,500/1,500).

Full matched WikiText-2 word perplexity (EleutherAI/wikitext_document_level, wikitext-2-raw-v1, no example limit; 2026-07-18): source 256.5985; this checkpoint 388.6980; absolute degradation +132.0995; relative degradation +51.48%. Independent native-logit scoring confirms the direction of degradation, but untemplated raw WikiText likelihood is not a suitable Gemma 4 instruction-model release gate. Investigation and raw evidence: releases/vlm-full-wikitext-perplexity/GEMMA4_PPL_INVESTIGATION.md and perplexity_results/vlm_full_wikitext/.

Reproduction

This directory contains the exact quantize.py, recipe.yaml, and versions.txt used for the artifact:

bash

python quantize.py \
  --model-path /path/to/source-model \
  --output-dir /path/to/new-quantized-model

Environment

text

Python: 3.13.2
torch: 2.11.0+cu128
CUDA: 12.8
transformers: 5.10.1
llm-compressor: 0.12.0
auto-round: 0.13.0
compressed-tensors: 0.17.1

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. LLiMa compiler output must remain separate unless its format and redistribution are approved. No Hugging Face upload is authorized for this batch.

Limitations

Quantization quality can vary by language, visual domain, prompt format, context length, and downstream runtime. MMStar does not replace application-specific validation. Audio weights and modality projectors remain in source precision. Users must validate compiler/runtime compatibility on the target deployment.