Modelpublic

EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly

sourceHugging Faceupdated 1mo agoView on Hugging Face

2likes1.2kdownloads

Model Card

Qwen3.8-27B-ABLITERATED — 3-bit MLX TextOnly

This is a 3-bit MLX TextOnly quantization of Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16, optimized for inference on Apple Silicon.

Quantization

—Format: MLX
—Quantization: 3-bit affine
—Group size: 64
—Vision tower: removed
—Runtime: mlx-lm
—Base model: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16

The language-model weights were converted to MLX and quantized to 3-bit. The vision components of the original multimodal model are not included, making this release TextOnly and reducing memory requirements.

Usage

Install MLX LM:

bash

pip install -U mlx-lm

Run an interactive chat:

bash

mlx_lm.chat \
  --model EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly

Or generate directly:

bash

mlx_lm.generate \
  --model EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly \
  --prompt "Hello! Explain quantization briefly." \
  --max-tokens 512

To disable thinking when supported by the model's chat template:

bash

mlx_lm.generate \
  --model EgorKodin/Qwen3.8-27B-ABLITERATED-3bit-MLX-TextOnly \
  --prompt "Hello!" \
  --max-tokens 512 \
  --chat-template-config '{"enable_thinking": false}'

Credits

Original model and fine-tune:

[Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16](https://huggingface.co/Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16)

This repository only provides an MLX quantization/conversion of the original model. All credit for the underlying model and abliteration work belongs to the original authors.