tommytracx/thoxroute

ThoxRoute — Hugging Face Space

Production Space: tommytracx/thoxroute.

The Space serves:

• —GET /healthz — liveness;
• —GET /runtimez — key-free runtime posture;
• —GET /v1/models — OpenAI-compatible model discovery;
• —POST /v1/chat/completions — OpenAI-compatible JSON/SSE;
• —POST /v1/route — native routing decision;
• —POST /api/chat — THOX Web Edition NDJSON; and
• —digital-human and speech surfaces from the composed service.

Production method

GitHub Actions is not the current release gate. Deploy from a tested local checkout as one atomic Space snapshot:

# 1. local source validation
python -m compileall -q thoxroute tests
pytest -q
bash -n serving/start-space.sh
bash -n serving/start-public-gateway.sh

docker build \
  -f huggingface-space/Dockerfile \
  -t thoxroute-space:0.9.3 \
  .

docker run --rm --entrypoint python thoxroute-space:0.9.3 -c \
  'from thoxroute.service import create_app; app=create_app(); print(app.version)'

# 2. clone production Space and replace it with the tested runtime snapshot
git clone git@hf.co:spaces/tommytracx/thoxroute /tmp/thoxroute-space
cd /tmp/thoxroute-space

git rm -r --ignore-unmatch .
cp -R /path/to/thoxroute/thoxroute ./thoxroute
cp -R /path/to/thoxroute/policies ./policies
cp -R /path/to/thoxroute/deploy ./deploy
cp -R /path/to/thoxroute/web ./web
cp -R /path/to/thoxroute/serving ./serving
cp /path/to/thoxroute/huggingface-space/Dockerfile ./Dockerfile
cp /path/to/thoxroute/huggingface-space/README.md ./README.md
cp /path/to/thoxroute/pyproject.toml ./pyproject.toml

git add -A
git commit -m 'deploy: ThoxRoute 0.9.3 resilient public gateway'
git push origin main

Do not commit provider credentials, Hugging Face tokens, or THOX API keys. Configure them under Settings → Variables and secrets.

Required production posture

The Docker image supplies these public-gateway defaults:

THOXROUTE_PUBLIC_MODEL_CATALOG=/app/policies/thoxllm_models.public.yaml
THOXROUTE_PROVIDER_READ_TIMEOUT_S=12
THOXROUTE_PROVIDER_THOX_RUSTCODER_READ_TIMEOUT_S=180
THOXROUTE_SSE_HEARTBEAT_S=10
THOXROUTE_OUTPUT_GUARD_ENABLED=true
THOXROUTE_EXPOSE_REASONING=false
THOXROUTE_STRIP_REASONING_TAGS=true
THOXROUTE_AUTO_CONTINUE=false
THOXROUTE_AUTO_CONTINUE_ROUNDS=0

The public entrypoint overrides a stale THOXROUTE_MODEL_CATALOG value with the tracked public profile.

Secrets commonly required by the production topology include:

• —HF_TOKEN — read access to the private ThoxRoute-1.5B GGUF;
• —THOXROUTE_API_KEYS — public application access keys;
• —THOXROUTE_OLLAMA_API_KEY — Ollama Cloud generation/fallback when used; and
• —any configured web-tool/provider credentials.

Runtime behavior

The API binds immediately. The local classifier warms in the background. Its watchdog proves a real Chat Completions response, not only Ollama daemon metadata. /runtimez reports classifier.primary_ready after the probe succeeds.

The public output guard hides internal reasoning, removes embedded reasoning tags, and caps visible output by route. The transport boundary contains upstream failures and completes SSE with one terminal [DONE].

Post-deploy acceptance

curl -fsS https://route.thox.ai/healthz | jq
curl -fsS https://route.thox.ai/runtimez | jq

Then run authenticated non-streaming and streaming checks from docs/OPENWEBUI_RESILIENCE_V2.md. Do not accept a deployment until the streaming client reads the body through EOF with exactly one [DONE].

OpenWebUI must remain configured with:

https://route.thox.ai/v1

See docs/OPERATIONS.md and docs/OPENWEBUI_RESILIENCE_V2.md for rollout, rollback, and troubleshooting.