ledercap/xai_voice_agent

1---2title: xAI Voice Agent3emoji: 🎙️4colorFrom: gray5colorTo: yellow6sdk: static7pinned: false8short_description: Your Reachy Mini as an xAI (Grok) realtime voice agent9tags:10 - reachy_mini11 - reachy_mini_python_app12---13 14# xAI Voice Agent for Reachy Mini15 16Turn your Reachy Mini into a live **xAI (Grok) realtime voice agent**. Talk to17your robot and it answers through its own speaker — leaning in while you speak,18lowering its antennae while it thinks, moving with its voice while it talks,19and performing physical **expressions** when the moment calls for it.20 21> Community app by LederCap — **not affiliated with xAI**. You bring your own22> xAI account, API key, and voice agent.23 24![Reachy Mini running xAI Voice Agent](assets/hero.jpg)25 26## See it in action27 28<video src="assets/demo.mp4" controls muted playsinline width="100%"></video>29 30## What you get31 32- **Always-listening conversation** — hands-free turn-taking via xAI server VAD,33  half-duplex (the mic is gated while the robot talks, so it never hears itself).34- **A robot that feels alive** — idle breathing, a lean-in while you speak, a35  thinking/searching antenna cue, head motion driven by the reply's audio.36- **Expressions** — the agent performs emotion moves (from Pollen's official37  emotions library) on its own via an `express()` tool, or you trigger them38  from one-tap chips in the control panel.39- **Live web & 𝕏 search** — optional; the agent answers with current40  information, fulfilled through xAI's Agent-Tools API.41- **A real control panel** — speaker/mic dials, turn-taking tuning, Interrupt,42  wake/sleep, credentials, and a live transcript.43 44## Requirements45 46- A **Reachy Mini** (developed and tested on the Wireless version — all AI47  compute runs in xAI's cloud, so the on-robot footprint is small).48- An **xAI account** with an API key and a **voice agent** — both free to49  create in the [xAI console](https://console.x.ai); API usage is billed by xAI50  at their rates.51 52## Quick start53 541. **Create a voice agent** in the [xAI console](https://console.x.ai). Its55   name, voice, and personality become your robot's — this app is the body;56   the agent you write is the soul. Copy the **agent id** (`agent_…`).572. **Get an xAI API key** from the same console.583. **Install this app** on your Reachy Mini from the dashboard and start it.594. Open the app's **gear icon** (the control panel), expand **Connection**,60   paste your API key and agent id, and hit **Save & Restart**.61 62The robot wakes, greets you, and listens. No key or agent id yet? The app63idles calmly with the panel available so you can add them — nothing crashes.64 65## The control panel66 67Open it from the dashboard's gear icon, or directly at68`http://<your-robot>:8042`.69 70![Control panel](assets/panel.png)71 72- **Speaker / Mic gain** dials — output volume and how strongly it hears you.73- **Sensitivity / End-of-turn** dials — live turn-taking tuning (how quiet is74  "speech", and how long a pause ends your turn).75- **Interrupt** — cuts the current reply off and reopens the mic.76- **Wake / Sleep** toggle, **web & 𝕏 search** toggle, **expression chips**,77  and a **live transcript** of what the agent is saying.78 79## Tips & troubleshooting80 81- **It doesn't say anything after starting** — almost always missing82  credentials: open the panel and check **Connection**.83- **Can't hear you?** Raise the **Mic gain** dial — the array is quiet.84- **Cuts you off mid-sentence?** Raise **End-of-turn** (silence ms) or lower85  **Sensitivity** toward 0.2 (lower threshold = hears quieter speech).86- **Rambling reply?** The header **Interrupt** button cuts it off.87- **Search feels slow** — live search takes ~30 s on xAI's side; the lowered88  antennae mean "working on it".89- Changing the agent id or key requires **Save & Restart** (the panel button).90 91## How it works92 93All AI compute runs in xAI's cloud — the robot only relays audio over WiFi and94runs the motion loop. Audio is PCM16 @ 16 kHz both ways (Reachy's native95rate). A single ~60 Hz control loop owns all motion, including wake/sleep and96expression playback. `web_search`/`x_search` are fulfilled via xAI's97Agent-Tools API; `express()` is fulfilled on-robot from the recorded-moves98library.99 100## Privacy101 102Your API key and agent id are stored only on your robot103(`~/.xai_voice_agent.env`, mode 0600) and are sent nowhere except `api.x.ai`.104Microphone audio streams to xAI's Realtime API while the app is running —105treat the robot like any other cloud voice assistant.106 107## Roadmap108 109- Full-duplex barge-in (interrupt by voice) — needs echo cancellation.110- Faster web search (raw-results provider + agent-side synthesis).111- Agent-less mode (no agent id required; instructions shipped with the app).112