Apppublic

Nethunter001/computer-use-agent

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes

App README

🖥️ AI Virtual Computer — HotBot × Bytebot

An AI agent that controls a real Linux computer. The HotBot multi-brain "racer" engine (GPT-5 / Gemini / Pollinations, fused for quality) is wired to a Bytebot virtual desktop (XFCE + Firefox + VS Code + terminal) so the agent can see the screen and operate the mouse, keyboard, apps and files — exactly like a human.

What's inside

Layer	Role
Bytebot desktop (`bytebotd`)	A full virtual computer + a `/computer-use` control API (mouse, keyboard, screenshot, apps, files), plus noVNC for a live view.
HotBot brains (`services/hotbot.js`)	The "racers" — several AI models run in parallel; the strongest reply wins.
Bridge (`services/bytebot.js`)	Thin client that lets the brains call the desktop's `/computer-use` API.
Agent loop (`services/agent.js`)	ReAct loop: the brain looks at a screenshot → emits one JSON action → it runs on the desktop → repeat until `finish`.
Web UI (`public/index.html`)	Chat to give the computer a task and watch it work on a live desktop.

How to use

1.Open the Space — you'll see the live desktop on the left and a control panel on the right.
2.In the 🤖 Agent tab, type a task in plain English, e.g.
3.open firefox and go to wikipedia.org
4.open the terminal and run "uname -a"
5.create a text file on the desktop with my notes
6.Watch the agent operate the computer step by step.

API

—GET /api/health — desktop + brain status
—GET /api/screenshot — current desktop PNG
—POST /api/agent — {"task":"open firefox and search openai"} → runs the agent loop
—POST /api/computer-action — {"action":"click","x":640,"y":400} or {"steps":[...]}
—POST /api/chat — {"message":"..."} → plain HotBot chat (no computer)

Configuration (Space → Settings → Variables)

Variable	Default	Purpose
`HOTBOT_MODEL`	`gpt-5`	Primary brain model.
`BYTEBOT_DESKTOP_URL`	`http://localhost:9990`	Desktop daemon URL.
`AGENT_MAX_STEPS`	`25`	Max actions per task.
`HOTBOT_SOLO`	(unset)	Force a single brain (`hotbot` or `gemini`).

Built on Bytebot + the HotBot engine.