yujiepan/awq-model-zoo
yujiepan/awq-model-zoo Here are some pre-computed awq information (scales & clips) used in llm-awq. Scripts Install the forked llm-awq at https://github.com/yujiepan-work/llm-awq/tree/a41a08e79d8eb3d6335485b3625410af22a74426. Note: works with transformers==4.35.2 Generating awq-info.pt: python do_awq.py --model_id mistralai/Mistral-7B-v0.1 --w_bit 8 --q_group_size 128 --dump_awq ./awq-info.pt Load a quantized model: You can use the offical repo to get a… See the full description on the dataset page: https://huggingface.co/datasets/yujiepan/awq-model-zoo.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face