FrescoHF/TQ3.5-27B-Musica-v1-Hybrid-GGUF

This is a hybrid quantization of the language model using Q5KM. The embeddings are quantized to q80, the first three and last three layers are in q80, and the fourth layers from both the beginning and the end are in q6_K. Since the embeddings, as well as the first and last layers, are the most sensitive to quantization, I decided to quantize them less. This will allow achieving higher-quality responses from the AI at the cost of a slight increase in weight. The quantization was done using llama.cpp version b8591 with this config:

^token_embd\.weight=q8_0
^output\.weight=q8_0
blk\.[0-2]\..*=q8_0
blk\.6[1-3]\..*=q8_0
blk\.3\..*=q6_K
blk\.60\..*=q6_K