Modelpublic

qwp4w3hyb/Mixtral-8x7B-Instruct-v0.1-iMat-GGUF

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face

0likes305downloads

Model Card

Mixtral-8x7B-Instruct-v0.1-iMat-GGUF

Source Model: mistralai/Mixtral-8x7B-Instruct-v0.1 Quantized with llama.cpp commit 46acb3676718b983157058aecf729a2064fc7d34 Imatrix was generated from the f16 gguf via this command:

./imatrix -c 512 -m $outpath/$basequantname -f $llamacpppath/groupsmerged.txt -o $out_path/imat-f16-gmerged.dat

Using the dataset from here