ddh0/GLM-4.5-Air-Derestricted-GGUF

GLM-4.5-Air-Derestricted-GGUF

This repository contains several custom GGUF quantizations of ArliAI/GLM-4.5-Air-Derestricted, to be used with llama.cpp.

The naming scheme for these custom quantizations is as follows:

`ModelName-DefaultType-FFN-UpType-GateType-DownType.gguf`

Where DefaultType refers to the default tensor type, and UpType, GateType, and DownType refer to the tensor types used for the ffn_up_exps, ffn_gate_exps, and ffn_down_exps tensors respectively.

Quantizations

These quantizations use Q80 for all tensors by default, including the dense FFN block. Only the conditional experts are downgraded. The shared expert is always kept in Q80. They were quantized using my own imatrix (the calibration text corpus can be found here).