Modelpublic

LeaderboardModel1/Qwythos-9B-v2-AutoRound-W4A16-Tuning

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes83downloads

quantization_config.json11 linesDownload Raw Back to root

1{2  "bits": 4,3  "data_type": "int",4  "group_size": 128,5  "sym": true,6  "low_gpu_mem_usage": true,7  "autoround_version": "0.14.2",8  "block_name_to_quantize": "model.language_model.layers",9  "quant_method": "auto-round",10  "packing_format": "auto_round:auto_gptq"11}