CoolFace
Apppublic

rmoxon/strands-siglip2-text-cpu

sourceHugging Faceupdated 4mo agoView on Hugging Face
0likes
App README

SigLIP2 Text Encoding Service (CPU)

Text-only sibling of `rmoxon/strands-embeddings-siglip2`.

Same model (google/siglip2-so400m-patch16-384), same 1152-dim output, same _unwrap + L2-norm math — but runs the text tower on CPU so search queries don't queue behind the GPU Space's image-batch backfill jobs.

Why this exists

The GPU Space serves both:

  1. 1.The backfill worker (process-siglip2-embeddings) — batches 200 images at a time
  2. 2.User search queries (embed-and-search-v2) — single text per call

ZeroGPU PRO permits only one concurrent GPU job per account, so a query that arrives mid-batch waits 5–9s for the GPU. Splitting query encoding onto a dedicated CPU Space eliminates that queue contention; warm queries land in ~500–800ms regardless of worker activity.

API endpoint

EndpointInputOutput
encode_texttext: str{embedding: float[1152], dimensions: 1152}
healthz(any){status, model, output_dim, device}

Use Gradio's /call/<api_name> SSE protocol (POST to submit, GET to stream results). See nextJS/supabase/functions/embed-and-search-v2/index.ts in the main Strands repo for the canonical client.

Hardware

  • —Runtime: CPU Basic (2 vCPU, 16 GB RAM, free tier)
  • —Model footprint: ~3.6 GB float32 (full AutoModel)
  • —Per-call latency (warm): ~500–800ms
  • —Cold start: ~10–20s after model load (Space sleeps after ~48h idle)

Embedding-space identity with the GPU Space is guaranteed by using identical model weights, identical processor config (padding="max_length", truncation=True, max_length=64), and identical L2 normalization.