Modelpublic

autoRiver/SAKI-InternVL3-2B-Instruct-bnb-4bit

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes7downloads

Model Card

Serialized BitsAndBytes NF4 4-bit checkpoint derived from OpenGVLab/InternVL3-2B-Instruct at revision f6c7b60375759170fd49f5e9e298e2178485c5ba.

json

{
  "bnb_4bit_compute_dtype": "bfloat16",
  "bnb_4bit_quant_type": "nf4",
  "bnb_4bit_use_double_quant": true,
  "load_in_4bit": true
}

Load with trust_remote_code=True, device_map="auto", and torch_dtype="auto".