Modelpublic

konkani/mistral-7b-instruct-konkani-awq

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes9downloads

Model Card

Mistral-7B-Instruct Konkani AWQ

This is the merged Konkani LoRA adapter quantized to 4-bit AWQ.

Source adapter: konkani/mistral-7b-instruct-konkani-lora

Quantization config:

json

{"zero_point": true, "q_group_size": 128, "w_bit": 4, "version": "GEMM"}

For vLLM, serve this checkpoint directly with --quantization awq.