sankarlabs/reachy-cerebras-test

Reachy Mini conversation app

Conversational app for the Reachy Mini robot combining realtime voice backends and choreographed motion libraries.

[image]

Table of contents

• —Overview
• —Architecture
• —Installation
• —Configuration
• —Running the app
• —LLM tools
• —Advanced features
• —Contributing
• —License

Overview

• —Real-time audio conversation loop for low-latency streaming, powered by the Hugging Face realtime backend using the built-in Hugging Face server or your own local endpoint.
• —Vision is handled by the realtime backend when the camera tool is used.
• —Layered motion system queues primary moves (dances, emotions, goto poses, breathing) while blending speech-reactive wobble.
• —Async tool dispatch integrates robot motion and camera capture. An optional web UI (--ui) provides personality selection, mic control, and settings.

Architecture

The app follows a layered architecture connecting the user, AI services, and robot hardware:

<p align="center"> <img src="docs/assets/conversationapparch.svg" alt="Architecture Diagram" width="600"/> </p>

Installation

[!IMPORTANT] Before using this app, you need to install Reachy Mini's SDK.<br> Windows support is currently experimental and has not been extensively tested. Use with caution.

<details open> <summary><b>Using uv (recommended)</b></summary>

Set up the project quickly using uv:

# macOS (Homebrew)
uv venv --python /opt/homebrew/bin/python3.12 .venv

# Linux / Windows (Python in PATH)
uv venv --python python3.12 .venv

source .venv/bin/activate
uv sync

Note: To reproduce the exact dependency set from this repo's uv.lock, run uv sync --frozen. This ensures uv installs directly from the lockfile without re-resolving or updating any versions.

Include dev dependencies:

uv sync --group dev

</details>

<details> <summary><b>Using pip</b></summary>

python -m venv .venv
source .venv/bin/activate
pip install -e .

Install dev dependencies:

pip install -e .[dev]                   # Development tools

</details>

Dependency groups

Configuration

The default setup uses the Hugging Face backend and does not require an API key.

Copy .env.example to .env when you want to point Hugging Face at your own local endpoint.

Hugging Face Connection Modes

Use the built-in Hugging Face server through the app-managed Space proxy. This is the default for a new install; set it explicitly only when you want to switch back from a saved local endpoint:

HF_REALTIME_CONNECTION_MODE=deployed

Run your own realtime voice backend using speech-to-speech on the same machine as the conversation app:

HF_REALTIME_CONNECTION_MODE=local
HF_REALTIME_WS_URL=ws://127.0.0.1:8765/v1/realtime

Run your own Hugging Face backend on your laptop and connect to it from Reachy Mini Wireless over the same Wi-Fi network:

HF_REALTIME_CONNECTION_MODE=local
HF_REALTIME_WS_URL=ws://<your-laptop-lan-ip>:8765/v1/realtime

For that LAN setup, make sure the backend listens on an address reachable from the robot, not only on 127.0.0.1.

If the backend stays bound to loopback on your laptop, you can forward it into the robot over SSH instead:

ssh -N -R 8765:127.0.0.1:8765 <robot-user>@<robot-host>

Then set this on the robot:

HF_REALTIME_CONNECTION_MODE=local
HF_REALTIME_WS_URL=ws://127.0.0.1:8765/v1/realtime

In the web UI's Settings view, the Connection section lets you choose either the built-in server or a local host:port target. The UI writes HF_REALTIME_CONNECTION_MODE for you, and the local path writes HF_REALTIME_WS_URL with a default of localhost:8765.

Running the app

Activate your virtual environment, then launch:

reachy-mini-conversation-app

[!TIP] Make sure the Reachy Mini daemon is running before launching the app. If you see a TimeoutError, it means the daemon isn't started. See Reachy Mini's SDK for setup instructions.

The app runs in console mode by default. Add --ui to also serve a web UI at http://127.0.0.1:7860/ for picking a personality, controlling the mic, and changing settings. All options are described in the CLI table below.

CLI options

Examples

# Audio-only conversation (no camera)
reachy-mini-conversation-app --no-camera

# Launch with the minimal web UI for personality/mic/settings control
reachy-mini-conversation-app --ui

LLM tools exposed to the assistant

[!NOTE] remember/forget facts are stored in memory.v1.json inside the app's instance data directory (~/.local/share/reachy_mini_conversation_app/ by default, or the instance path used by the desktop launcher). forget only removes facts matched by query. To reset all remembered facts, delete this file.

Advanced features

Built-in motion content is published as open Hugging Face datasets:

• —Emotions: `pollen-robotics/reachy-mini-emotions-library`
• —Dances: `pollen-robotics/reachy-mini-dances-library`

<details> <summary><b>Custom profiles</b></summary>

Create custom profiles with dedicated instructions and enabled tools.

For normal usage, select a profile from the UI and save it for startup. That selection is persisted in startup_settings.json.

If no startup settings have been saved yet, you can still seed startup from the environment with REACHY_MINI_CUSTOM_PROFILE=<name> to load profiles/<name>/. If neither is set, the default profile is used.

Each profile should include instructions.txt (prompt text). If that file is missing or empty, the app logs a warning and falls back to profiles/default/instructions.txt. greeting.txt is optional and controls how the robot should start the conversation after the backend connects. tools.txt (list of allowed tools) is recommended. If missing for a non-default profile, the app falls back to profiles/default/tools.txt. Profiles can optionally contain custom tool implementations.

Startup greeting:

On startup, once the realtime backend is connected and ready, the app sends the active profile's greeting.txt as an internal text turn so the model opens with a fresh spoken greeting. Keep this file as a short instruction, not a fixed script, for example:

Greet me warmly in one sentence, in character, and vary the wording each time.

If greeting.txt is missing, the app uses the built-in default greeting prompt.

Enabling tools:

List enabled tools in tools.txt, one per line. Prefix with # to comment out:

play_emotion
# move_head

# My custom tool defined locally
sweep_look

Tools are resolved first from Python files in the profile folder (custom tools), then from the core library src/reachy_mini_conversation_app/tools/ (like dance, camera). Installed Hugging Face Space tools can also be enabled here after you add them with tool-spaces.

Custom tools:

On top of built-in tools found in the core library, you can implement custom tools specific to your profile by adding Python files in the profile folder. Custom tools must subclass reachy_mini_conversation_app.tools.core_tools.Tool (see that module for the interface).

Edit personalities from the UI:

When running with --ui, the Home view lists available profiles (folders under profiles/) plus the built-in default:

• —Tap a card to apply that personality and start talking.
• —Tap "Custom" to create a new personality by entering a name, instructions, and an optional startup greeting prompt. It copies tools.txt from the default profile and stores the files under user_personalities/<name>/ in the app instance directory (next to .env/startup_settings.json).

Note: switching a personality reloads its instructions and tools in place via a quick backend reconnect — no app restart. Editing the active profile's files on disk needs a re-select (or restart) to apply.

</details>

<details> <summary><b>Locked profile mode</b></summary>

To create a locked variant of the app that cannot switch profiles, edit src/reachy_mini_conversation_app/config.py and set the LOCKED_PROFILE constant to the desired profile name:

LOCKED_PROFILE: str | None = "mars_rover"  # Lock to this profile

When LOCKED_PROFILE is set, the app always uses that profile, ignoring saved startup settings, REACHY_MINI_CUSTOM_PROFILE, and the web UI. The UI shows "(locked)" and disables all profile editing controls. This is useful for creating dedicated clones of the app with a fixed personality. Clone scripts can simply edit this constant to lock the variant.

</details>

<details> <summary><b>External profiles and tools</b></summary>

You can extend the app with profiles/tools stored outside the repository defaults.

• —Core profiles are under profiles/.
• —Core tools are under src/reachy_mini_conversation_app/tools/.

Recommended layout:

external_content/
├── external_profiles/
│   └── my_profile/
│       ├── instructions.txt
│       ├── greeting.txt     # optional startup greeting prompt
│       ├── tools.txt        # optional (see fallback behavior below)
│       └── voice.txt        # optional
├── external_tools/
│   └── my_custom_tool.py
└── installed_tool_spaces.json

Environment variables:

Set these values in your .env when you want env-driven external profile/tool selection:

# Optional fallback/manual profile selector:
REACHY_MINI_CUSTOM_PROFILE=my_profile
REACHY_MINI_EXTERNAL_PROFILES_DIRECTORY=./external_content/external_profiles
REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY=./external_content/external_tools
# Optional convenience mode:
# AUTOLOAD_EXTERNAL_TOOLS=1

Loading behavior:

• —Default/strict mode: tools.txt defines enabled tools explicitly. Every name in tools.txt must resolve to either a built-in tool (src/reachy_mini_conversation_app/tools/) or an external tool module in REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY.
• —Convenience mode (AUTOLOAD_EXTERNAL_TOOLS=1): all valid *.py tool files in REACHY_MINI_EXTERNAL_TOOLS_DIRECTORY are auto-added.
• —External profile fallback: if the selected external profile has no tools.txt, the app falls back to built-in profiles/default/tools.txt.
• —Duplicate safety: every loaded tool class must expose a unique Tool.name. The app now fails fast if two tool implementations claim the same tool name.

This supports both:

1. 1.Local external tools used with built-in/default profile.
2. 2.Local external profiles used with built-in default tools.

</details>

<details> <summary><b>Hugging Face Space tools</b></summary>

You can install MCP-compatible Hugging Face Spaces as remote tool sources for this app. Private Spaces work too, as long as HF_TOKEN is set (or you have run hf auth login) for an account that can access them.

# install + enable in active profile
reachy-mini-conversation-app tool-spaces add <owner/space-name>

# enable in a specific profile
reachy-mini-conversation-app tool-spaces add <owner/space-name> --profile NAME

# install without enabling
reachy-mini-conversation-app tool-spaces add <owner/space-name> --install-only

# list installed spaces
reachy-mini-conversation-app tool-spaces list

# remove an installed space
reachy-mini-conversation-app tool-spaces remove owner/space-name

The app validates the Space slug through the Hugging Face Hub, probes the standard MCP endpoint (sending the HF token only to private Spaces), discovers tools, enables them in the active profile's tools.txt, and writes the installed Space to:

• —installed_tool_spaces.json in the managed app instance directory
• —external_content/installed_tool_spaces.json in terminal mode

Recommended tags for discoverability on Hugging Face:

• —reachy-mini-tool
• —mcp

These tags are advisory only. Installation still relies on successful MCP validation, not on tag presence.

</details>

<details> <summary><b>Multiple robots on the same subnet</b></summary>

If you run multiple Reachy Mini daemons on the same network, use:

reachy-mini-conversation-app --robot-name <name>

<name> must match the daemon's --robot-name value so the app connects to the correct robot.

</details>

Contributing

We welcome bug fixes, features, profiles, and documentation improvements. Please review our contribution guide for branch conventions, quality checks, and PR workflow. Working with an AI coding assistant? Point it at `AGENTS.md` — it codifies our engineering standards for agents.

Quick start:

• —Fork and clone the repo
• —Follow the installation steps (include the dev dependency group)
• —Run contributor checks listed in CONTRIBUTING.md

License

Apache 2.0