NoNameFound404/sentence-transformer-embeddings

Sentence-Transformer Embedding Service

A small HTTP microservice that hosts `sentence-transformers/all-MiniLM-L6-v2` and returns 384-dimensional, L2-normalized embeddings. FastAPI in a Docker container, deployed on Hugging Face Spaces.

It is a drop-in replacement for embedding done in-process by the AIVIS-ADO RAG backend: same model, same dimension, same normalization — so vectors stay compatible with the existing Postgres/pgvector data.

API

• —/embed mirrors AIVIS-ADO embed_text; /embed/batch mirrors embed_batch (default batch_size=64).
• —All vectors are L2-normalized (normalize_embeddings=True).
• —/ redirects to /docs (interactive API docs).

Authentication

When the API_KEY env var is set, /embed and /embed/batch require a matching X-API-Key header. When it is empty (the local default), the endpoints are open. /health is always public, so health checks and uptime pings work.

curl -X POST https://<space>.hf.space/embed \
  -H "Content-Type: application/json" \
  -H "X-API-Key: <your-key>" \
  -d '{"text": "hello world"}'

Run locally

Python 3.12 (matches the container).

python -m venv .venv
.\.venv\Scripts\Activate.ps1        # Windows;  source .venv/bin/activate on macOS/Linux
pip install -r requirements.txt
uvicorn app.main:app --reload --port 8005

The model downloads into ./model_cache on first start (~90 MB). Then:

curl http://localhost:8005/health
curl -X POST http://localhost:8005/embed -H "Content-Type: application/json" -d '{"text":"hello world"}'

Deploy

The repo is the Space — push and it rebuilds:

git push space HEAD:main
hf spaces logs <user>/<space> --build --follow

The Dockerfile installs deps, bakes the model into the image, and serves uvicorn on port 7860. API_KEY lives as a Space secret (never in the repo). Full walkthrough: DEPLOY_HUGGINGFACE.md.

Hardware & concurrency

Free cpu-basic = 2 vCPU / 16 GB. Memory is not the constraint; CPU is.

• —--workers 2 — one per vCPU. Each worker loads its own ~500 MB model copy (16 GB has room), but going past the vCPU count buys nothing for CPU-bound embedding.
• —Don't flood it. Embedding is CPU-bound, so N concurrent requests each take ~N/2 × their solo time. Callers should cap concurrency (~2) and batch via /embed/batch rather than firing many requests at once — otherwise latency climbs without throughput improving.
• —Free Spaces sleep after ~48 h idle; ping /health on a schedule to keep one warm.

Use it from AIVIS-ADO

Set EMBEDDING_SERVICE_URL and EMBEDDING_API_KEY in the AIVIS-ADO environment, then call it over the existing httpx dependency:

import asyncio, os, httpx

_client = httpx.AsyncClient(
    base_url=os.environ["EMBEDDING_SERVICE_URL"],
    headers={"X-API-Key": os.environ["EMBEDDING_API_KEY"]},
    timeout=120.0,          # seeding batches are slow on 2 vCPUs; don't retry into the fire
)
_sem = asyncio.Semaphore(2)  # cap concurrency at the Space's vCPU count


async def async_embed_text(text: str) -> list[float]:
    async with _sem:
        r = await _client.post("/embed", json={"text": text})
    r.raise_for_status()
    return r.json()["embedding"]          # 384-dim, L2-normalized


async def async_embed_batch(texts: list[str], batch_size: int = 64) -> list[list[float]]:
    async with _sem:
        r = await _client.post("/embed/batch", json={"texts": texts, "batch_size": batch_size})
    r.raise_for_status()
    return r.json()["embeddings"]

Keep the graceful-degradation pattern: on connection errors / 5xx, raise EmbeddingsUnavailableError so callers fall back to empty results instead of crashing.

Project layout

app/
  config.py    # env vars: API_KEY, MODEL_NAME, MODEL_CACHE_DIR
  model.py     # cached model singleton + embed_one() / embed_many()
  schemas.py   # request/response models
  main.py      # FastAPI app, routes, API-key dependency, request-timing log
Dockerfile     # installs deps, bakes the model, serves on 7860
requirements.txt