Modelpublic

Steven0090/llava1.6-Mistral-7B-Instruct-v0.2-gguf

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

2likes50downloads

Model Card

This is Q8_0 quantization model of Llava1.6.

Run it by llama_cpp

python

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Steven0090/llava1.6-Mistral-7B-Instruct-v0.2-gguf",
	filename="Mistral-7B-Instruct-v0.2-Q8_0.gguf",
)