CoolFace
Modelpublic

vanch007/Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16

sourceHugging Faceupdated 6mo agoView on Hugging Face
1likes180downloads
Model Card

Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16

MLX-VLM conversion of huihui-ai/Huihui-gemma-4-26B-A4B-it-abliterated.

Overview

  • —Format: MLX-VLM
  • —Precision: bf16
  • —Size: 48G
  • —Source model type: Gemma4ForConditionalGeneration
  • —Source pipeline: any-to-any
  • —Intended runtime: mlx-vlm and LM Studio

Conversion Notes

  • —Converted for local Apple Silicon inference with mlx-vlm
  • —Kept in MLX-VLM multimodal layout for image-text generation
  • —Includes local config compatibility fixes required during conversion/debugging

Validation

Local checks on Apple Silicon:

  • —model loading in mlx-vlm: passed
  • —local conversion / quantization: passed
  • —text generation smoke test: mixed
  • —image generation path smoke test: mixed
  • —lm studio 0.4.11+1 loading: not supported yet
  • —notes: Conversion finished successfully, but Gemma 4 runtime behavior still depends on current mlx-vlm and LM Studio support. In local checks, the models load in mlx-vlm and can emit tokens, but output quality still needs tuning, and LM Studio 0.4.11+1 currently refuses to load Gemma 4.

Files

Important files in this repo:

  • —config.json
  • —generation_config.json
  • —chat_template.jinja
  • —processor_config.json
  • —tokenizer.json
  • —tokenizer_config.json
  • —model.safetensors.index.json
  • —model-*.safetensors

Usage

Text generation

bash
mlx_vlm.generate \
  --model /path/to/Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16 \
  --prompt "Describe local inference in one short sentence." \
  --max-tokens 128 \
  --temperature 1.0 \
  --trust-remote-code

Image prompt

bash
mlx_vlm.generate \
  --model /path/to/Huihui-gemma-4-26B-A4B-it-abliterated-mlx-bf16 \
  --image /path/to/example.png \
  --prompt "Describe this image." \
  --max-tokens 128 \
  --temperature 1.0 \
  --trust-remote-code

LM Studio

LM Studio 0.4.11+1 detects these repos but currently refuses to load Gemma 4 with ValueError: Gemma 4 support is not ready yet, stay tuned!. Use mlx-vlm for now until LM Studio adds native Gemma 4 support.

Notes

This repo reflects a local conversion workflow and validation pass on Apple Silicon. Behavior can vary with mlx-vlm version, sampling parameters, and prompt style.