Modelpublic

Atomheart-Father/Nyx-RP-9B-Instruct-2608-v1-MLX-4bit

sourceHugging Faceapache-2.0updated 6d agoView on Hugging Face

0likes29downloads

Model Card

Nyx-RP-9B-Instruct-2608-v1-MLX-4bit

MLX 4-bit quantization of Indexnusrefather/Nyx-RP-9B-Instruct-2608-v1 (Qwen3.5-9B roleplay finetune), converted from GGUF (mradermacher/Nyx-RP-9B-Instruct-2608-v1-GGUF) with a bounded-memory GGUF → MLX transcoder.

—Quantization: 4-bit, group size 64 (affine)
—Model size: ~4.7 GB
—Context length: 262144

Usage

Requires a recent mlx-lm that supports the qwen3_5 model type:

bash

pip install -U mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("Atomheart-Father/Nyx-RP-9B-Instruct-2608-v1-MLX-4bit")
response = generate(model, tokenizer, prompt="hello", verbose=True)

For chat, use the tokenizer's chat template:

python

from mlx_lm import load, generate

model, tokenizer = load("Atomheart-Father/Nyx-RP-9B-Instruct-2608-v1-MLX-4bit")
messages = [{"role": "user", "content": "hello"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
response = generate(model, tokenizer, prompt=prompt, verbose=True)