EnlistedGhost/Mistral-Small-4-119B-2603-GGUF

<img src="https://huggingface.co/EnlistedGhost/Mistral-Small-4-119B-2603-GGUF/resolve/main/resources/Introducing%20Mistral-Small-4.png" alt="Example image" width="398" height="338">

Mistral-Small-4-119B-A6B | GGUF Edition

<br />

Coversion Details (Safetensors --> GGUF)

• —Converter: Llama.cpp (Build 9888)
• —Quantizer: Llama.cpp (Build 9840)

Info<br />___

Thank You for Viewing This Release! <br /> Model files (GGUF) are still being uploaded.<br /> This modelcard will be updated soon! Your understanding and patience is very much appreciated.

• —Mistral-Small-4-119B-A6B is a huge model in physical storage size and due to this: Uploads are taking longer than expected.* <br /><br />

Technical Details<br />___________________<br />These quantized GGUF files found in this release are unique<br />in the sense that when initially converting this model it was observed that there are native Float32 (F32) weights in the layers.<br /> Upon realizing this during the initial conversion;the model was then re-converted using an "F32" flag in Llama.cpp (Version 9888) prior to being quantized.<br /> (This was done in order to snub quality loss compared to converting to BF16 or F16 conversion+quantize)