impost0r/Gemma-4-31B-Cognitive-Unshackled-mlx-8bit

Gemma-4-31B-Cognitive-Unshackled-mlx-8bit

8-bit MLX quantization of aifeifei798/Gemma-4-31B-Cognitive-Unshackled, for inference on Apple Silicon with mlx-lm.

• —Format: MLX, 8-bit (8.500 bits/weight)
• —Runtime dtype: bfloat16
• —Backbone converted: text only (gemma4_text). The source is a vision-language model (Gemma4ForConditionalGeneration); vision and audio components are not included in this conversion.
• —Size: ~30 GB on disk; peak ~33 GB RAM during generation.

Usage

pip install mlx-lm
mlx_lm.generate --model impost0r/Gemma-4-31B-Cognitive-Unshackled-mlx-8bit \
  --prompt "Explain unified memory on Apple Silicon."

from mlx_lm import load, generate

model, tokenizer = load("impost0r/Gemma-4-31B-Cognitive-Unshackled-mlx-8bit")
messages = [{"role": "user", "content": "Hello!"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

Notes

• —The base model is an uncensored ("Cognitive-Unshackled") fine-tune; outputs are minimally filtered. Use responsibly.
• —License inherited from the Gemma base model — see the Gemma Terms of Use.

Conversion

Converted with the transformers-to-mlx skill using mlx_lm.convert (--q-bits 8). Validated by generation-coherence, output-dtype (bfloat16, no float32 contamination), and long-sequence (2000-token) stability tests. Gemma 4 is natively supported in mlx-lm (PR #1093 and follow-ups); no model code changes were required.