Modelpublic

darthcrawl/artifex-rp-orpheus-llama-3.1-8b-mlx-6bit

sourceHugging Facellama3.1updated 5mo agoView on Hugging Face

0likes39downloads

Model Card

artifex-rp-orpheus-llama-3.1-8b-mlx-6bit

Part of the [Artifex RP](https://huggingface.co/collections/darthcrawl) series.

MLX 6-bit (group-size 32) quantization of artifex-rp-orpheus-llama-3.1-8b for Apple Silicon.

For full model details, sample outputs, and training notes → see the main model card.

Quickstart

bash

pip install mlx-lm
mlx_lm.generate --model darthcrawl/artifex-rp-orpheus-llama-3.1-8b-mlx-6bit \
  --system-prompt "Write in vivid, restrained, intelligent prose. Concrete sensory detail. Never pretentious. Tension carries more weight than description. Match the user's energy and length." \
  --prompt "She hadn't seen him in three years." \
  --temp 0.85 --top-p 0.95 --max-tokens 400

Quantization

Parameter	Value
Bits	6
Group size	32
Effective bits/weight	7.0
Size	6.6GB

Available variants

Repo	Size	Fits on
artifex-rp-orpheus-llama-3.1-8b	~16GB	24GB+ Mac (BF16)
artifex-rp-orpheus-llama-3.1-8b-mlx-8bit	8.4GB	16GB Mac
artifex-rp-orpheus-llama-3.1-8b-mlx-6bit	6.6GB	16GB Mac
artifex-rp-orpheus-llama-3.1-8b-mlx-4bit-dwq	4.7GB	8GB Mac

License

Weights inherit the base model license. See artifex-rp-orpheus-llama-3.1-8b.