CoolFace
Apppublic

Thox-ai/thoxvoice-chat

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes
App README

ThoxVoice Chat

This header card configures the Hugging Face Space (Docker SDK). The Space is a hosted demo/mirror; the source of truth is the private GitHub repo `ttracx/thoxvoice-chat`. Model-stack keys are provided via Space secrets (never hardcoded) — see Deploy on Hugging Face Spaces.

THOX.ai's on-device agent-teams realtime conversation product. Talk (voice, soon video) with a team of THOX agents in one live session — running on your own machine through THOX model stacks, cloud strictly optional.

Forked from `smolagents/hf-realtime-voice` and reshaped from a single-assistant, voice-only, remote-backend demo into a local-first, multi-agent, voice+video product.

Status: MVP vertical slice — AI layer + local realtime backend + re-skinned frontend, 23 headless tests green. Real STT/TTS and LiveKit video are the next phases. See `docs/CUSTOMIZATION_PLAN.md`.

What's different from upstream

UpstreamThoxVoice Chat
ModelsRemote HF speech-to-speech load balancer (required)On-device (Ollama/vLLM), cloud opt-in
Model layerFixed pipeline`thoxcore` router — swappable backends via env
AgentsOne assistantAgent teams — multiple THOX agents, one session
MediaVoice onlyVoice + video (LiveKit, Phase 2)
BrandHugging FaceTHOX.ai (dark-first, emerald)
PostureHosted, meteredLocal-first, private by default

The browser client's OpenAI-Realtime-GA WebSocket protocol is kept verbatim — the THOX customization sits behind it (the directUrl seam). See `docs/adr/0002-thox-realtime-seam.md`.

Quick start (local, headless-friendly)

bash
# 1. Python env (kept off C: to save space)
python -m venv .venv
.venv\Scripts\python -m pip install -r thox/requirements.txt   # Windows
# source .venv/bin/activate && pip install -r thox/requirements.txt  # POSIX

# 2. Run the test suite (no models, no GPU needed — uses EchoBackend)
.venv\Scripts\python -m pytest -q          # -> 23 passed

# 3. Run the server (serves the UI + the local realtime backend)
.venv\Scripts\python -m uvicorn thox.realtime.server:app --port 8080

# 4. Open http://localhost:8080 , open Settings, set the server URL to
#    `localhost:8080` (the client appends /v1/realtime), and start talking/typing.

With no environment configured, the stack falls back to the deterministic EchoBackend so it always answers. Start Ollama (ollama serve, default llama3.1) and it's used automatically — local-first, no config.

Point the model stack somewhere else (swap = env, not code)

bash
export THOX_OLLAMA_MODEL=llama3.1                          # on-device default
export THOX_OPENAI_BASE_URL=http://localhost:8000/v1       # vLLM / thoxllm-factory
export THOX_ANTHROPIC_API_KEY=sk-ant-...                   # cloud fallback (opt-in)
export THOX_PRIORITY=ollama,openai,anthropic,echo          # force routing order

Architecture at a glance

Browser (upstream client, THOX skin)
   │  OpenAI Realtime GA over WebSocket  (directUrl -> localhost)
   v
thox/realtime  --  RealtimeSession -- TeamOrchestrator (who speaks?)
   │                     │
   │                     v
thox/core      --  ThoxRouter (thoxcore) -- Ollama - vLLM - Anthropic - Echo
  • —`thox/core` — the model-stack seam: typed router + swappable backends, local-first fallback.
  • —`thox/agents` — agent teams: personas + turn routing (mention / affinity / round-robin).
  • —`thox/realtime` — local realtime backend speaking the client's protocol; STT/TTS seam.

Full diagrams: `docs/ARCHITECTURE.md` · portfolio view: `docs/ecosystem_map.md`.

The agent team (default)

AgentRoleRoutes on
NovaHost — greets, keeps flow, hands offgreetings, "team", "who"
SableResearch — technical / how-to / code"why", "how", "error", "explain"
VeraPlanning — steps, decisions, summaries"plan", "steps", "next", "summary"

Address one directly with @Sable … or Vera: …; otherwise the orchestrator picks by skill affinity, then round-robin. Define your own team via TeamOrchestrator / ThoxAgent.

Tests

bash
.venv\Scripts\python -m pytest -q     # 23 tests: router fallback, team routing, full WS turn loop

All headless — no GPU, no models, no network — via the EchoBackend.

Repo layout

See `docs/ARCHITECTURE.md`. Upstream docs are preserved at `docs/UPSTREAM_README.md`, `CONTEXT.md`, and `DESIGN.md`.

Deploy on Hugging Face Spaces

This repo is Space-ready (Docker SDK, header card at the top of this README). The Space is a hosted demo/mirror — the source of truth is the GitHub repo.

  1. 1.The Dockerfile runs thox.realtime.server:app on port 7860.
  2. 2.With no config, the Space boots on the deterministic EchoBackend.
  3. 3.To enable real model responses, add Space secrets/variables (never commit keys):
KindKeyExamplePurpose
secretTHOX_ANTHROPIC_API_KEYsk-ant-…Cloud LLM (opt-in)
secretTHOX_OPENAI_API_KEY…Auth for a remote vLLM/thoxllm-factory endpoint
variableTHOX_OPENAI_BASE_URLhttps://…/v1Remote OpenAI-compatible endpoint
variableTHOX_ENABLE_OLLAMA0No local Ollama on a Space — skip its probe
variableTHOX_PRIORITYanthropic,openai,echoRouting order
bash
   hf spaces secrets add   <owner>/thoxvoice-chat --secrets THOX_ANTHROPIC_API_KEY=sk-ant-...
   hf spaces variables add <owner>/thoxvoice-chat --env THOX_ENABLE_OLLAMA=0
  1. 1.Hardware: the MVP (voice text-turn loop) runs on the free/basic CPU tier. Phase-1.5 local STT/TTS and Phase-2 LiveKit video will want a GPU tier (e.g. t4-small+) — set it then, not now.

Credits

  • —Upstream client & audio pipeline: `smolagents/hf-realtime-voice` (Pollen Robotics × Hugging Face), OpenAI Realtime GA protocol.
  • —THOX layer, agent teams, and thoxcore router: THOX.ai.