HyperCluster/Fara-BrowserUse

1# FaraCUA Backend
2
3The backend server for FaraCUA - a Computer Use Agent (CUA) demo powered by Microsoft's Fara-7B vision-language model and Modal for serverless GPU inference.
4
5## Overview
6
7This backend provides:
8
9- **WebSocket API** - Real-time communication with the React frontend for streaming agent actions
10- **REST API** - Model listing, random question generation, and trace storage
11- **FARA Agent Integration** - Runs the Fara agent with Playwright for browser automation
12- **Modal Integration** - Proxies requests to Modal's vLLM endpoint and trace storage
13
14## Architecture
15
16```
17┌─────────────┐     WebSocket      ┌─────────────┐     HTTP      ┌─────────────┐
18│   Frontend  │ ◄───────────────► │   Backend   │ ◄───────────► │    Modal    │
19│   (React)   │                    │  (FastAPI)  │               │   (vLLM)    │
20└─────────────┘                    └─────────────┘               └─────────────┘
21                                         │
22                                         │ Playwright
23                                         ▼
24                                   ┌─────────────┐
25                                   │   Browser   │
26                                   │  (Headless) │
27                                   └─────────────┘
28```
29
30## Files
31
32| File | Description |
33|------|-------------|
34| `server.py` | Main FastAPI server with WebSocket and REST endpoints |
35| `modal_fara_vllm.py` | Modal deployment for vLLM inference and trace storage |
36| `pyproject.toml` | Python dependencies |
37| `.env.example` | Example environment configuration |
38
39## Setup
40
41### 1. Install Dependencies
42
43```bash
44# Using uv (recommended)
45uv sync
46
47# Or using pip
48pip install -e .
49```
50
51### 2. Install Playwright
52
53```bash
54playwright install chromium
55```
56
57### 3. Deploy Modal Endpoints
58
59```bash
60modal deploy backend/modal_fara_vllm.py
61```
62
63This deploys:
64- **vLLM Server** - GPU-accelerated inference for Fara-7B at `https://<workspace>--fara-vllm-serve.modal.run`
65- **Trace Storage** - Endpoint for storing task traces at `https://<workspace>--fara-vllm-store-trace.modal.run`
66
67### 4. Configure Environment
68
69Copy `.env.example` to `.env` and fill in your values:
70
71```bash
72cp .env.example .env
73```
74
75Required variables:
76
77| Variable | Description |
78|----------|-------------|
79| `FARA_MODEL_NAME` | Model name (default: `microsoft/Fara-7B`) |
80| `FARA_ENDPOINT_URL` | Modal vLLM endpoint URL (from deploy output) |
81| `FARA_API_KEY` | API key (default: `not-needed` for Modal) |
82| `MODAL_TOKEN_ID` | Modal proxy auth token ID |
83| `MODAL_TOKEN_SECRET` | Modal proxy auth token secret |
84| `MODAL_TRACE_STORAGE_URL` | Modal trace storage endpoint URL |
85
86Get Modal proxy auth tokens at: https://modal.com/settings/proxy-auth-tokens
87
88### 5. Run the Server
89
90```bash
91# Development mode
92uvicorn backend.server:app --host 0.0.0.0 --port 8000 --reload
93
94# Or directly
95python -m backend.server
96```
97
98## API Endpoints
99
100### WebSocket
101
102- `ws://localhost:8000/ws` - Real-time agent communication
103  - **Receives**: `user_task`, `stop_task`, `ping`
104  - **Sends**: `heartbeat`, `agent_start`, `agent_progress`, `agent_complete`, `agent_error`
105
106### REST
107
108| Method | Endpoint | Description |
109|--------|----------|-------------|
110| GET | `/api/health` | Health check |
111| GET | `/api/models` | List available models |
112| GET | `/api/random-question` | Get a random example task |
113| POST | `/api/traces` | Store a trace (proxies to Modal) |
114
115## Trace Storage
116
117Task traces are automatically uploaded to Modal volumes for research purposes. Traces include:
118
119- Task instruction and model used
120- Step-by-step agent actions with screenshots
121- Token usage and timing metrics
122- User evaluation (success/failed)
123
124Duplicate traces (same ID and instruction) are automatically overwritten to capture the latest evaluation.
125
126## Docker
127
128The backend is designed to run in Docker alongside the frontend. See the root `Dockerfile` for the combined deployment.
129
130```bash
131# Build from root
132docker build -t fara-cua .
133
134# Run with env file
135docker run -d --name fara-cua -p 7860:7860 --env-file backend/.env fara-cua
136```
137
138## Development
139
140### Running Locally
141
142For local development, you can run the backend separately:
143
144```bash
145cd backend
146uvicorn server:app --host 0.0.0.0 --port 8000 --reload
147```
148
149Make sure the frontend is configured to connect to `http://localhost:8000`.
150
151### Testing Modal Endpoints
152
153```bash
154# Test vLLM endpoint
155modal run backend/modal_fara_vllm.py::test
156
157# Check deployment status
158modal app list
159```
160
161## License
162
163See the root LICENSE file for license information.