CoolFace
Modelpublic

nawta/Heron-NVILA-Lite-2B-F16-GGUF

sourceHugging Faceapache-2.0updated 10mo agoView on Hugging Face
0likes17downloads
Model Card

Heron-NVILA-Lite-2B GGUF (F16)

This is the F16 (full precision) GGUF version of turing-motors/Heron-NVILA-Lite-2B-hf, a Vision-Language Model optimized for Japanese.

Model Details

Model Architecture

ComponentDetails
Vision EncoderSigLIP (paligemma-siglip-so400m-patch14-448)
Projectormlpdownsample2x2_fix
LLMQwen2.5-1.5B
Total Parameters~2B

Requirements

This model requires the mmproj (multimodal projector) file to process images:

Available Quantizations

QuantizationFile SizeUse Case
F163.3 GBMaximum quality
Q8_01.8 GBHigh quality
Q6_K1.4 GBBalanced
Q5_K_M1.2 GBBalanced
Q4_K_M1.1 GBEfficient
Q3_K_M0.9 GBCompact
Q2_K0.7 GBMinimum size

Usage with llama.cpp

bash
# Download the model and mmproj
wget https://huggingface.co/nawta/Heron-NVILA-Lite-2B-F16-GGUF/resolve/main/Heron-NVILA-Lite-2B-F16.gguf
wget https://huggingface.co/nawta/Heron-NVILA-Lite-2B-mmproj-GGUF/resolve/main/mmproj-heron-nvila-lite-2b-f16.gguf

# Run inference
./llama-mtmd-cli \
  -m Heron-NVILA-Lite-2B-F16.gguf \
  --mmproj mmproj-heron-nvila-lite-2b-f16.gguf \
  --image your_image.jpg \
  -p "この画像について説明してください。"

Memory Requirements

QuantizationLLMmmprojTotal
F163.3 GB807 MB~4.1 GB
Q8_01.8 GB807 MB~2.6 GB
Q6_K1.4 GB807 MB~2.2 GB
Q5KM1.2 GB807 MB~2.0 GB
Q4KM1.1 GB807 MB~1.9 GB
Q3KM0.9 GB807 MB~1.7 GB
Q2_K0.7 GB807 MB~1.5 GB

License

This model inherits the license from the original Heron-NVILA-Lite-2B model. Please refer to turing-motors/Heron-NVILA-Lite-2B-hf for license details.

Acknowledgments