DarrenJiaImbue/gemma-4-E4B-it-qat-litertlm

Gemma 4 E4B IT (QAT) — LiteRT-LM bundle, community conversion

Unofficial conversion of google/gemma-4-E4B-it-qat-mobile-transformers to LiteRT-LM .litertlm format, produced entirely from the public QAT mobile checkpoint with a patched litert-torch (adds the quantized-safetensors import path requested in google-ai-edge/litert-torch#998 and #1044).

No weights were taken from the official prebuilt litert-community/gemma-4-E4B-it-litert-lm; it was used only for verification. The chat template is the LiteRT-compatible template from that repo (the checkpoint's own template uses Jinja constructs the runtime's minijinja engine cannot render).

Fidelity vs the official prebuilt

The QAT quantization is preserved bit-exactly (verified tensor-by-tensor against the official bundle):

• —embedder + all 42 per-layer-embedder INT2 tables: byte-identical
• —prefilldecode int2/int4/int8 weights: **343/344 identical** including scales (the exception, `perlayermodelprojection`, is stored unquantized in the public checkpoint)
• —Remaining differences (norm gammas etc.) exist because the public checkpoint stores float tensors in bf16 while the official bundle was built from f32 masters — unrecoverable from public data.

Quality (35-question auto-scored benchmark, greedy, litert-lm 0.13.1)

30/35 outputs are token-identical to the official prebuilt on CPU.

Contents & configuration

Text-only (no vision/audio/drafter sections). prefill_128/prefill_1024 signatures, cache length 4096, f32 KV cache, dynamic-range int2/int4/int8 weights (gemma4_qat_w248 recipe), ~3.25 GB.

Usage

# pip install litert-lm
import litert_lm

engine = litert_lm.Engine("model.litertlm", backend=litert_lm.Backend.CPU())
conv = engine.create_conversation(
    sampler_config=litert_lm.SamplerConfig(top_k=1))
print(conv.send_message("What is the capital of France?"))

Gemma is provided under and subject to the terms of the upstream model's license; see the base model card.