CoolFace
Apppublic

Agents-MCP-Hackathon/pictionagent-reachy

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
5likes
App README

Pictionagent (feat Reachy)

About

Pictionagent is an interactive draw-and-guess game developed during the Gradio Agents & MCP Hackathon (June 2025). In this game, an AI agent draws concepts on a canvas while users try to guess what's being drawn.

🔗 Related Projects

  • —Drawing Generation Server: concept-to-drawing-points, another gradio hugging face space, connected as a MCP server, for the generation of the drawing image and points.

👥 Hackathon team

  • —thervier (solo participant)

🎥 Demo Videos

Features

  • —🤖 AI-powered drawing agent
  • —🎮 Multi-user real-time gameplay
  • —🏆 Live leaderboard
  • —🎯 Multiple drawing categories
  • —⚡ Real-time drawing animation
  • —🔄 Configurable game settings

Reachy or not Reachy?

The name and title of this space imply that this game involves Reachy, the robot developed by Pollen Robotics and Hugging Face. Unfortunately, due to technical constraints on Hugging Face Docker spaces, I was unable to deploy the docker with Reachy robot here on Hugging Face spaces.

However, the complete experience with Reachy is available through local deployment. All the necessary code is included in this repository, and you can find the setup instructions below to run the full version with Reachy integration with Docker.

A collective experience

Pictionagent is designed as a collective experience where all players connect to the same instance. This shared environment allows everyone to see the same changes, in real-time, creating an exciting competitive atmosphere where players race to be the first to guess correctly.

Game Overview

🎮 How to Play

  1. 1.Start a Round
  2. 2.Select a topic (Shapes, Animals, or Famous Monuments)
  3. 3.Click the start button to begin
  1. 1.Watch & Guess
  2. 2.Watch as the AI agent draws
  3. 3.Submit your guess using the chat interface
  4. 4.Points are awarded for correct guesses
  1. 1.Round End
  2. 2.The correct answer is revealed
  3. 3.See who guessed correctly
  4. 4.Start a new round!

📱 Interface Tabs

TabDescription
🎮 GameMain gameplay interface
👥 Connection InfosView connected users and set username
⚙️ Game ConfigAdjust drawing speed and point count
🏆 LeaderboardView top players

Technical implementation

🏗️ Architecture

The project consists of three main components:

  1. 1.reachy_api.py
  2. 2.Manages Reachy robot API, using Reachy2 SDK (provided by Pollen Robotics)
  3. 3.Includes fake robot mode (so that the full experience still works if a Reachy robot is not connected, in real or simulated in a Docker container)
  1. 1.pictionagent.py
  2. 2.Core game logic and state management
  3. 3.Handles agents, game flow and scoring
  1. 1.app.py
  2. 2.Gradio application interface
  3. 3.Manages user interactions
  4. 4.Instantiates a Pictionagent class, that will be shared as a singleton between all the applications connected to the same instance

🤖 Game agent

The game experience is managed by an agent, implemented with Langgraph ReAct agent with OpenAI gpt-4o-mini model.

The flow of the experience is handled dynamically by the agent from this simple prompt, that does not explicitly describe the steps:

You are a drawing agent and we are playing a game. I will give you a topic, and you will choose randomly an item to draw that will correspond to the given topic (for example, if the topic is "Animal", the item could be "Dog", "Cat", "Bird", etc.). Do not always choose first the obvious like Elephant or Eiffel Tower and do not choose the same item twice.
The user can only guess during the drawing, after the drawing is completed the user can no longer guess, so do not expect them to, just display the drawing image using provided tool and share the item drawn.
If it was guessed, share what it was and the id of the user who guessed it (if it was guessed).

and the following tools:

  • —set_status: Updates the current status of the agent (implemented locally in Pictionagent)
  • —save_item: Saves the chosen item to draw (implemented locally in Pictionagent)
  • —generate_image_and_points: Generates the image and points to draw (implemented on another gradio HF space, connected as a MCP server)
  • —draw: Draws the points on the canvas (implemented locally in Pictionagent)
  • —display_drawing_image_and_item: Displays the initial image and chosen item to draw (implemented locally in Pictionagent)

💭 Guess system

The guessing part is managed separately by a simple Chat, using Langgraph and ChatOpenAI with model gpt-4o-mini and the following prompt:

We are playing a draw and guess game. You have drawn an item and the user tries to guess it.
You must return a message indicating if the user has guessed the item correctly or not, but never tell the item to guess to the user if they have not guessed it correctly.
If the user is very close (for example with some small typos), you can validate the guess.

Answer using json format following this structure:
{
    "guessed": true/false,
    "response": "response to the user"
}

Workaround for reactivity

Initially, the draw tool had the list of points to draw as input argument. However, the game agent had difficulties calling this tool with the whole list of points (it was working but took more than 20 seconds to generate the call to the tool, which was not acceptable for the user experience). As a workaround, we added a solution to save the list of points when received from the generate_image_and_points tool and reuse them directly in the draw tool, which is now triggered without expecting the points as input argument.

Build and deploy the experience with Reachy robot

As explained before, we have implemented everything to run with Reachy2 robot in a docker image containing our Gradio app and all elements needed to run a ROS simulation of Reachy2 (as provided by Pollen Robotics in docker image https://hub.docker.com/r/pollenrobotics/reachy2). It uses a nginx server to ensure all needed elements are accessible on port 8080.

However, there are some constraints for HF Docker spaces that prevented us from deploying a working version of the docker image on HF space.

All the code needed for this is available in this repository so you can run it in Docker directly.

To do so:

  1. 1.Make sure you have Docker installed and working
  2. 2.Clone the repository of this space
  3. 3.Move to the root of this repo
  4. 4.Build the Docker image:
bash
   docker build -t pictionagent-reachy .
  1. 1.Run the container:
bash
   docker run --rm -e USE_HTTP_HOST=1 -e OPENAI_API_KEY=<your_openai_api_key> -p 8080:8080 pictionagent-reachy

The application will be available in your browser at: http://localhost:8080/

License

This project is licensed under the Apache 2.0 License.

Acknowledgments

Thanks to:

  • —Hugging Face for hosting the space
  • —Pollen Robotics team for making the Reachy2 robot easily accessible via their docker image
  • —Pollen Robotics team for reachy2_emotions repository with APIs and contents to play animations on Reachy
  • —Gradio for the nice framework
  • —All people involved in the organization of Gradio Agents & MCP Hackathon