Modelpublic

pierretokns/SmolVLM2-2.2B-Instruct-GGUF

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face

0likes95downloads

Model Card

SmolVLM2-2.2B-Instruct-GGUF (Q4KM)

Q4KM quantized GGUF conversion of SmolVLM2-2.2B-Instruct.

Files

File	Size	Description
`SmolVLM2-2.2B-Instruct-Q4_K_M.gguf`	1.0 GB	Main model (Q4KM quantized)
`mmproj-SmolVLM2-2.2B-Instruct-f16.gguf`	832 MB	Vision projector (F16)

Usage

With Ollama

dockerfile

FROM ./SmolVLM2-2.2B-Instruct-Q4_K_M.gguf
PROJECTOR ./mmproj-SmolVLM2-2.2B-Instruct-f16.gguf
PARAMETER num_ctx 4096
PARAMETER temperature 0.1

With llama.cpp

bash

./llama-mtmd-cli -m SmolVLM2-2.2B-Instruct-Q4_K_M.gguf --mmproj mmproj-SmolVLM2-2.2B-Instruct-f16.gguf --image screenshot.png -p "Click the Submit button"

License

Apache 2.0