Apppublic

gatilin/nanoVLM-222M-webui

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes

App README

nanoVLM-222M Web UI

A Hugging Face Spaces deployment of nanoVLM-222M — a lightweight 222M parameter Vision-Language Model built in pure PyTorch.

🚀 Live Demo: Hugging Face Spaces

Model Overview

Spec	Value
Parameters	222M
Vision Encoder	SigLIP-B/16 (85M)
Language Decoder	SmolLM2-135M
Modality Projection	Pixel Shuffle + Learned Projection
MMStar Accuracy	35.3%
Training Dataset	1.7M samples from The Cauldron
Training Time	~6 hours on a single H100

Features

—🖼️ Image Upload — Drag & drop or click to upload images
—💬 Interactive Chat — Multi-turn conversation with image context
—⚡ Fast Inference — Optimized for CPU and GPU with KV-cache
—🎛️ Generation Controls — Adjust temperature, top-k, top-p, max tokens
—📱 Mobile Friendly — Responsive Gradio interface

Local Development

Setup

bash

# Clone the repository
git clone https://huggingface.co/spaces/gatilin/nanoVLM-222M-webui
cd nanoVLM-222M-webui

# Install dependencies
pip install -r requirements.txt

# Launch the app
python app.py

Docker

bash

docker build -t nanovlm-webui .
docker run -p 7860:7860 nanovlm-webui

Usage

Web Interface

1.Upload an image (JPG, PNG, WEBP, HEIC supported)
2.Type your question about the image
3.Click Send or press Enter
4.Adjust generation parameters in the sidebar for different outputs

API

python

import gradio_client

client = gradio_client.Client("gatilin/nanoVLM-222M-webui")
result = client.predict(
    image="path/to/image.jpg",
    prompt="What is in this image?",
    temperature=0.7,
    max_new_tokens=200
)
print(result)

Architecture

nanoVLM-222M
├── Vision Encoder (SigLIP-B/16)
│   └── Patch Embeddings → ViT Blocks → Layer Norm
├── Modality Projector
│   └── Pixel Shuffle → Linear Projection
└── Language Decoder (SmolLM2-135M)
    ├── Token Embeddings + Image Embeddings
    ├── RoPE + RMSNorm + GQA Attention
    └── Causal LM Head → Autoregressive Generation

Citation

bibtex

@misc{wiedmann2025nanovlm,
  author = {Luis Wiedmann and Aritra Roy Gosthipaty and Andrés Marafioti},
  title = {nanoVLM},
  year = {2025},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/huggingface/nanoVLM}}
}

License

Apache-2.0