almernzh/Gemma-4-12B-it-AWQ-INT4-Multimodal-LongContext

Gemma-4-12B-it-AWQ-INT4-Multimodal-LongContext

AWQ INT4 weights for google/gemma-4-12B-it, prepared for lower VRAM use while keeping the base model practical for text, image, audio, structured output, and long-context experiments.

Quantization

• —Method: AWQ
• —Weight format: INT4
• —Compute scheme: W4A16
• —Calibration samples: 128
• —Max calibration length: 4096 tokens
• —Calibration style: document QA, OCR-style text, structured JSON output, coding prompts, table and chart descriptions, transcript-style prompts, and long-context retrieval prompts
• —Tooling: LLM Compressor 0.12.0, Transformers 5.10.1, PyTorch 2.11.0+cu128
• —GPU used: NVIDIA A100-SXM4-80GB

Local Check

A short generation check was run after saving the weights.

• —Test prompt: return a JSON object with status: ok
• —Result: passed
• —Peak VRAM during local load test: 22.98 GB

Notes

These weights are intended for practical inference tests with lower memory use than the original precision. Quantization can affect exact wording, numeric precision, and long-context recall. Full 256K context behavior should only be assumed after testing in your own setup.