zeromodels/siglip2_base_p16_384

*See [our collection](https://huggingface.co/collections/zeromodels/siglip2-6a8eaf393fde5bbc3359a5e8) for all versions of SigLIP 2.*

Run SigLIP 2 with Keras 3: JAX, PyTorch, or TensorFlow

  

zeromodels/siglip2basep16_384

Paper: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features (arXiv:2502.14786) · HF Papers

SigLIP 2 keeps SigLIP's sigmoid loss and adds captioning-based pretraining, self-distillation, and masked prediction for stronger dense features. It uses a 256k multilingual Gemma vocabulary, so many languages work without a separate multilingual checkpoint.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of `google/siglip2-base-patch16-384` for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a zero-shot image-text checkpoint (SigLIP2ZeroShotClassify): pass image(s) and text prompts at inference time.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.siglip2 import (
    SigLIP2Processor,
    SigLIP2ZeroShotClassify,
)

processor = SigLIP2Processor.from_weights("zeromodels/siglip2_base_p16_384")
model = SigLIP2ZeroShotClassify.from_weights("zeromodels/siglip2_base_p16_384")

labels = [
    "a photo of a cat",
    "a photo of a dog",
    "a photo of a car",
    "a photo of a living room",
]
inputs = processor(text=labels, image_paths="your_image.jpg")
output = model(
    {
        "images": inputs["images"],
        "token_ids": inputs["input_ids"],
    }
)
print(output["image_logits"].shape)

Load any SigLIP 2 variant the same way with from_weights("zeromodels/<variant>"):

Tips

• —Set KERAS_BACKEND before importing Keras / zeromodels.
• —Prefer Processor.from_weights(...) so image size and tokenizer match the variant.
• —Map processor input_ids to model token_ids. No padding mask is required.
• —Tokenizer is Gemma-based (multilingual); prefer Processor.from_weights.
• —See SigLIP 2 docs and Loading Weights.
• —Community / upstream safetensors still work via the hf: prefix, e.g. SigLIP2ZeroShotClassify.from_weights("hf:google/siglip2-base-patch16-384").

Special Thanks

A huge thank you to the Google SigLIP 2 authors for creating and releasing these models.

License: Apache 2.0.