CoolFace
Modelpublic

Chemin-AI/Mixtral-8x7B-Instruct-v0.1-gguf

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face
1likes153downloads
Model Card

# Supa-AI/Mixtral-8x7B-Instruct-v0.1-gguf This model was converted to GGUF format from `mistralai/Mixtral-8x7B-Instruct-v0.1` using llama.cpp. Refer to the original model card for more details on the model.

## Available Versions

  • —Mixtral-8x7B-Instruct-v0.1.q4_0.gguf (q4_0)
  • —Mixtral-8x7B-Instruct-v0.1.q4_1.gguf (q4_1)
  • —Mixtral-8x7B-Instruct-v0.1.q5_0.gguf (q5_0)
  • —Mixtral-8x7B-Instruct-v0.1.q5_1.gguf (q5_1)
  • —Mixtral-8x7B-Instruct-v0.1.q8_0.gguf (q8_0)
  • —Mixtral-8x7B-Instruct-v0.1.q3_k_s.gguf (q3KS)
  • —Mixtral-8x7B-Instruct-v0.1.q3_k_m.gguf (q3KM)
  • —Mixtral-8x7B-Instruct-v0.1.q3_k_l.gguf (q3KL)
  • —Mixtral-8x7B-Instruct-v0.1.q4_k_s.gguf (q4KS)
  • —Mixtral-8x7B-Instruct-v0.1.q4_k_m.gguf (q4KM)
  • —Mixtral-8x7B-Instruct-v0.1.q5_k_s.gguf (q5KS)
  • —Mixtral-8x7B-Instruct-v0.1.q5_k_m.gguf (q5KM)
  • —Mixtral-8x7B-Instruct-v0.1.q6_k.gguf (q6_K)

## Use with llama.cpp Replace FILENAME with one of the above filenames.

### CLI:

bash
                llama-cli --hf-repo Supa-AI/Mixtral-8x7B-Instruct-v0.1-gguf --hf-file FILENAME -p "Your prompt here"

### Server:

bash
                llama-server --hf-repo Supa-AI/Mixtral-8x7B-Instruct-v0.1-gguf --hf-file FILENAME -c 2048

## Model Details