Modelpublic

ncterry1/local-llm-lab

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes2downloads

quantization_config.json6 linesDownload Raw Back to root

1{2  "load_in_4bit": true,3  "bnb_4bit_quant_type": "nf4",4  "bnb_4bit_compute_dtype": "float16",5  "bnb_4bit_use_double_quant": true6}