Modelpublic

cyboghostginx/Dolphin-X1-8B-mlx-4Bit

sourceHugging Facellama3.1updated 26d agoView on Hugging Face

0likes725downloads

Model Card

Dolphin-X1-8B-mlx-4Bit

4-bit MLX conversion of dphn/Dolphin-X1-8B for Apple silicon. 4.5 GB, single shard, converted with mlx-lm 0.26.4. Weights only, no fine-tuning.

bash

pip install mlx-lm
mlx_lm.generate --model cyboghostginx/Dolphin-X1-8B-mlx-4Bit \
                --prompt "your prompt" --max-tokens 512

python

from mlx_lm import load, generate

model, tokenizer = load("cyboghostginx/Dolphin-X1-8B-mlx-4Bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "hello"}], tokenize=False, add_generation_prompt=True
)
print(generate(model, tokenizer, prompt=prompt, verbose=True))

License

Built with Llama. Governed by the Llama 3.1 Community License, inherited through the base model. Safety behaviour follows the Dolphin base, so it answers requests a stock instruct model declines. You are responsible for how you deploy it.