OsaurusAI/gemma-4-12B-it-MXFP8

<p align="center"> <a href="https://osaurus.ai"><img src="./osaurus-x-banner.png" alt="Osaurus AI"></a> </p>

<h3 align="center">Gemma 4 12B-it - MXFP8 (MLX)</h3> <p align="center">Apple Silicon MLX bundle for Osaurus and compatible vMLX runtimes.</p>

<p align="center"> <a href="https://osaurus.ai"><img src="https://img.shields.io/badge/Web-osaurus.ai-blue" alt="Website"></a>&nbsp; <a href="https://huggingface.co/OsaurusAI"><img src="https://img.shields.io/badge/HF-OsaurusAI-yellow?logo=huggingface" alt="OsaurusAI"></a>&nbsp; <a href="https://github.com/jjang-ai/jangq"><img src="https://img.shields.io/badge/GitHub-JANG_Source-blue?logo=github" alt="JANG source"></a> </p>

Important update (2026-06-03 4:06 PM PDT): These weights were rebuilt with the verified Gemma 4 12B fix. If you downloaded this repository before 2026-06-03 4:06 PM PDT, delete the local copy and re-download.

Model Details

Runtime Notes

These rebuilt bundles preserve the tied token embedding in fp16 while keeping the main projection weights quantized. This fixes the bad prior artifact where embed_tokens.weight was packed and scaled like a normal linear weight.

The bundle includes generation_config.json, chat_template.jinja, tokenizer_config.json, and processor_config.json for Osaurus/vMLX loading.

Loading

Use Osaurus for local Apple Silicon chat and multimodal workflows, or load the bundle in a compatible MLX runtime:

from mlx_lm import load, generate

model, tokenizer = load("OsaurusAI/gemma-4-12B-it-MXFP8")
print(generate(model, tokenizer, "Hello", max_tokens=128))

Verification

Local release check for this rebuild: