jsyeom/awq-kivi-w4a16kv8

AWQ W4A16 + KIVI-INT8 KV (Llama-3.1-8B-Instruct) w4_awq_llama_31_8b/ — W4 weights The 4-bit weights of the AWQ W4A16 model (asymmetric / standard AWQ, group size 128). Stored per layer: layer_0.safetensors … layer_31.safetensors + embeddings.safetensors. The 7 linears per layer are quantized (packed int4 codes + fp16 per-group scale + uint8 per-group zero); everything else stays fp16: key dtype shape *.qweight (2 int4 codes / byte; lo nibble = even… See the full description on the dataset page: https://huggingface.co/datasets/jsyeom/awq-kivi-w4a16kv8.