Modelpublic

RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w4a16

sourceHugging Facellama3.1updated 2y agoView on Hugging Face

33likes8.8kdownloads

quantize_config.json13 linesDownload Raw Back to root

1{2  "bits": 4,3  "group_size": 128,4  "damp_percent": 0.01,5  "desc_act": true,6  "static_groups": false,7  "sym": true,8  "true_sequential": true,9  "model_name_or_path": null,10  "model_file_base_name": "model",11  "quant_method": "gptq",12  "checkpoint_format": "gptq"13}