Modelpublic

baa-ai/Gemma-4-31B-it-RAM-3bit-MLX

sourceHugging Facegemmaupdated 5mo agoView on Hugging Face

2likes180downloads

Model Card

Gemma-4-31B-it — RAM 3bit (MLX)

A quantized build of google/gemma-4-31B-it produced by baa.ai. Retains the full vision tower, unlike other pre-quantized MLX variants of this model.

Property	Value
Size on disk	19.49 GB
Format	MLX
Base model	google/gemma-4-31B-it
Vision tower	Retained

Usage

python

from mlx_vlm import load, generate

model, processor = load("baa-ai/Gemma-4-31B-it-RAM-3bit-MLX")
prompt = processor.tokenizer.apply_chat_template(
    [{"role": "user", "content": [{"type": "text", "text": "Hello!"}]}],
    add_generation_prompt=True, tokenize=False,
)
result = generate(model, processor, prompt, max_tokens=512, verbose=True)
print(result.text)

Benchmark results

Measured on vanilla MMLU (500 questions) and MathVision MCQ (20 questions). Unsloth Gemma 4 MLX variants strip the vision tower — they cannot process images.

[image]

This model vs Unsloth Gemma-4-31B 3bit (500-question MMLU):

—RAM: 89.2%  ·  Unsloth: 75.6%  ·  Gap: +13.6 pp
—Size: Unsloth 19.52 GB → RAM 19.49 GB (30 MB smaller, vision retained)
—MathVision: RAM 50.0%  ·  Unsloth: N/A (no vision)

<details> <summary>Full results table — all 31B variants</summary>

80-question run

Bits	Unsloth MMLU	RAM MMLU	GAP	RAM Vision
8bit	70.7%	90.0%	+19.3	55.0%
4bit	73.8%	86.2%	+12.4	60.0%
3bit	72.5%	86.2%	+13.7	50.0%

500-question run

Bits	Unsloth MMLU	RAM MMLU	GAP
8bit	71.8%	90.4%	+18.6
4bit	71.8%	89.0%	+17.2
3bit	75.6%	89.2%	+13.6

</details>

License

Inherited from the upstream Gemma license.

[baa.ai](https://baa.ai)

Black Sheep AI Products

[Shepherd](https://baa.ai/shepherd.html) — Private AI deployment platform that shrinks frontier models by 50-60% through RAM compression, enabling enterprises to run sophisticated AI on single GPU instances or Apple Silicon hardware. Deploy in your VPC with zero data leaving your infrastructure. Includes CI/CD pipeline integration, fleet deployment across Apple Silicon clusters, air-gapped and sovereign deployment support, and multi-format export (MLX, GGUF). Annual cloud costs from ~$2,700 — or run on a Mac Studio for electricity only.

[Watchman](https://baa.ai/watchman.html) — Capability audit and governance platform for compressed AI models. Know exactly what your quantized model can do before it goes live. Watchman predicts which capabilities survive compression in minutes — replacing weeks of benchmarking. Includes compliance-ready reporting for regulated industries, quality valley warnings for counterproductive memory allocations, instant regression diagnosis tracing issues to specific tensors, and 22 adversarial security probes scanning for injection, leakage, hallucination, and code vulnerabilities.

Learn more at [baa.ai](https://baa.ai) — Sovereign AI.