Modelpublic

darthcrawl/mistral-7b-instruct-v0.3-artisan-mlx-4bit-dwq

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes34downloads

Model Card

mistral-7b-instruct-v0.3-artisan — MLX 4-bit DWQ

4-bit MLX quantization of darthcrawl/mistral-7b-instruct-v0.3-artisan.

python

from mlx_lm import load, generate
model, tok = load("darthcrawl/mistral-7b-instruct-v0.3-artisan-mlx-4bit-dwq")
print(generate(model, tok, "hello", max_tokens=64))