DJLougen/Harmonic-Hermes-9B-MLX-8bit

Harmonic-Hermes-9B — MLX 8bit

MLX 8-bit (group size 64, affine) build of DJLougen/Harmonic-Hermes-9B — the Stage-2 agentic, tool-calling fine-tune of Harmonic-9B (Qwen 3.5 9B). Built for MLX (Apple Silicon, and CUDA via the MLX CUDA backend).

Approximate size: ~8.9 GB — Near-lossless 8-bit. Runs comfortably on 12 GB+ unified memory.

Usage

pip install mlx-lm
mlx_lm.generate --model DJLougen/Harmonic-Hermes-9B-MLX-8bit \
  --prompt "You can call get_weather(location). What's the weather in Toronto?"

from mlx_lm import load, generate
model, tokenizer = load("DJLougen/Harmonic-Hermes-9B-MLX-8bit")
messages = [{"role": "user", "content": "List three primes, then verify each."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

The model uses <think> reasoning blocks and emits tool calls in the Hermes/Qwen function-calling format. The chat template (with tool-use support) ships with this repo.

Conversion

Converted from DJLougen/Harmonic-Hermes-9B to MLX with mlx-lm 0.31.2 on an NVIDIA GB10 (DGX Spark) using the MLX CUDA backend. This is a text-generation (LLM) build; the vision tower is not included — for multimodal inference use the GGUF mmproj in DJLougen/Harmonic-Hermes-9B-GGUF.

Variants

License

Apache 2.0 — same as the base model.