CoolFace
Apppublic

ledercap/xai_voice_agent

sourceHugging Faceupdated 2mo agoView on Hugging Face
1likes
App README

xAI Voice Agent for Reachy Mini

Turn your Reachy Mini into a live xAI (Grok) realtime voice agent. Talk to your robot and it answers through its own speaker — leaning in while you speak, lowering its antennae while it thinks, moving with its voice while it talks, and performing physical expressions when the moment calls for it.

Community app by LederCap — not affiliated with xAI. You bring your own xAI account, API key, and voice agent.

[image]

See it in action

<video src="assets/demo.mp4" controls muted playsinline width="100%"></video>

What you get

  • —Always-listening conversation — hands-free turn-taking via xAI server VAD, half-duplex (the mic is gated while the robot talks, so it never hears itself).
  • —A robot that feels alive — idle breathing, a lean-in while you speak, a thinking/searching antenna cue, head motion driven by the reply's audio.
  • —Expressions — the agent performs emotion moves (from Pollen's official emotions library) on its own via an express() tool, or you trigger them from one-tap chips in the control panel.
  • —Live web & 𝕏 search — optional; the agent answers with current information, fulfilled through xAI's Agent-Tools API.
  • —A real control panel — speaker/mic dials, turn-taking tuning, Interrupt, wake/sleep, credentials, and a live transcript.

Requirements

  • —A Reachy Mini (developed and tested on the Wireless version — all AI compute runs in xAI's cloud, so the on-robot footprint is small).
  • —An xAI account with an API key and a voice agent — both free to create in the xAI console; API usage is billed by xAI at their rates.

Quick start

  1. 1.Create a voice agent in the xAI console. Its name, voice, and personality become your robot's — this app is the body; the agent you write is the soul. Copy the agent id (agent_…).
  2. 2.Get an xAI API key from the same console.
  3. 3.Install this app on your Reachy Mini from the dashboard and start it.
  4. 4.Open the app's gear icon (the control panel), expand Connection, paste your API key and agent id, and hit Save & Restart.

The robot wakes, greets you, and listens. No key or agent id yet? The app idles calmly with the panel available so you can add them — nothing crashes.

The control panel

Open it from the dashboard's gear icon, or directly at http://<your-robot>:8042.

[image]

  • —Speaker / Mic gain dials — output volume and how strongly it hears you.
  • —Sensitivity / End-of-turn dials — live turn-taking tuning (how quiet is "speech", and how long a pause ends your turn).
  • —Interrupt — cuts the current reply off and reopens the mic.
  • —Wake / Sleep toggle, web & 𝕏 search toggle, expression chips, and a live transcript of what the agent is saying.

Tips & troubleshooting

  • —It doesn't say anything after starting — almost always missing credentials: open the panel and check Connection.
  • —Can't hear you? Raise the Mic gain dial — the array is quiet.
  • —Cuts you off mid-sentence? Raise End-of-turn (silence ms) or lower Sensitivity toward 0.2 (lower threshold = hears quieter speech).
  • —Rambling reply? The header Interrupt button cuts it off.
  • —Search feels slow — live search takes ~30 s on xAI's side; the lowered antennae mean "working on it".
  • —Changing the agent id or key requires Save & Restart (the panel button).

How it works

All AI compute runs in xAI's cloud — the robot only relays audio over WiFi and runs the motion loop. Audio is PCM16 @ 16 kHz both ways (Reachy's native rate). A single ~60 Hz control loop owns all motion, including wake/sleep and expression playback. web_search/x_search are fulfilled via xAI's Agent-Tools API; express() is fulfilled on-robot from the recorded-moves library.

Privacy

Your API key and agent id are stored only on your robot (~/.xai_voice_agent.env, mode 0600) and are sent nowhere except api.x.ai. Microphone audio streams to xAI's Realtime API while the app is running — treat the robot like any other cloud voice assistant.

Roadmap

  • —Full-duplex barge-in (interrupt by voice) — needs echo cancellation.
  • —Faster web search (raw-results provider + agent-side synthesis).
  • —Agent-less mode (no agent id required; instructions shipped with the app).