flow1254/flow-browser-agent

Selfapi-v2 — Headless Chrome + Unpacked Extensions

A Hugging Face Space that runs a persistent, headless Chrome for Testing browser inside Docker with three custom unpacked extensions auto-loaded, plus a live web monitor to watch the browser in real time.

The browser stays logged in across restarts — on Hugging Face the profile is snapshotted to a Postgres database (see Persistence below); on a self-hosted Docker setup it lives in a named volume. The bundled extensions sync cookies / automate workflows for ChatGPT, Google Gemini, and Google Flow.

Bundled extensions

Both are Manifest V3 extensions with background service workers. Each subfolder also contains the matching backend server.

Backend servers

Each extension is the browser half of a standalone project. The extension uses your authenticated browser session and forwards requests to a local API server (which you run yourself — see each subproject's README for setup):

✅ The HF Space now runs all three servers in the same container as Chrome, and the monitor doubles as a single API gateway that fronts them on one public port — see API gateway. (The extensions dial their servers on hardcoded 127.0.0.1:9225/9226/9227, so the servers must share Chrome's localhost — a separate Space can't reach them.) docker-compose.yml does the same wiring for self-hosting.

How it works

HF Space (port 3001)
        │
        ▼
┌─────────────────────────────────────────────────────────┐
│ Docker container (Debian 12)                             │
│                                                          │
│  profilesync ──► restores profile from Postgres on boot  │
│   (Go)          and backs it up every 5 min / on exit    │
│                   │                                      │
│  monitor (3-tab) ─┤  serves the live UI + /chrome.log    │
│   (Go, :3001)     │                                      │
│                   ▼                                      │
│  Chrome for Testing 145 (--headless=new)                 │
│    --load-extension=/opt/gpt,/opt/gemini,/opt/flow       │
│    --remote-debugging-port=9222  (CDP)                   │
│    --user-data-dir=/home/chrome/data  (persistent)       │
│                   ▲                                      │
│  socat :9223 ─────┘  forwards external CDP → 127.0.0.1   │
└─────────────────────────────────────────────────────────┘

start_hf.sh is the boot sequence:

1. 1.Restore profile — if DATABASE_URL is set, profilesync restore pulls the last Chrome profile snapshot from Postgres so logins survive an HF restart/rebuild.
2. 2.Profile repair & cleanup — fixes a nested data/data layout and removes cache/junk to keep the persistent profile small.
3. 3.Lock removal — clears Singleton* / LOCK files left by an unclean shutdown.
4. 4.Preferences injection — enables extensions.ui.developer_mode and sets a clean exit_type so the "restore pages?" popup never appears.
5. 5.Monitor server — starts the Go 3-tab monitor on :3001 and symlinks chrome.log.
6. 6.CDP proxy — socat exposes the DevTools port (9223 → 127.0.0.1:9222).
7. 7.Auto-backup — if DATABASE_URL is set, profilesync backup runs every 5 min (plus a final backup on SIGTERM/SIGINT shutdown).
8. 8.Launch Chrome — Chrome for Testing in --headless=new with all three extensions.

Persistence (staying logged in)

Hugging Face Spaces on the free `cpu-basic` tier have no persistent disk — the container filesystem (including /home/chrome/data) is wiped on every restart, rebuild, or factory reboot. To keep logins, the profile is snapshotted to an external Postgres database.

How it works (the profilesync Go tool, built into the image):

• —On boot: restores the profile from Postgres (profilesync restore).
• —Every 5 min + on shutdown: backs the profile up (profilesync backup).
• —Stores it in one row: chrome_profile(id, updated_at, data bytea) — table auto-created.
• —All of this is gated on the `DATABASE_URL` env var. If it is not set, restore/backup are silently skipped and logins are lost on restart.

Setup — one-time

1. 1.Create a free Postgres at neon.tech → new project → copy the connection string. It looks like:

   postgresql://USER:PASSWORD@HOST.neon.tech/DBNAME?sslmode=require

1. 1.Add it as a Space secret: HF Space → Settings → Variables and secrets → New secret
2. 2.Name: DATABASE_URL
3. 3.Value: the connection string from step 1
4. 4.Saving the secret auto-restarts the Space. Because nothing was backed up yet, the first boot has nothing to restore — so log in once after the secret is set. Within 5 minutes that login is snapshotted to Postgres and will survive all future restarts.

💡 Verify a backup landed: connect to the DB and run SELECT id, updated_at, octet_length(data) FROM chrome_profile; — a row with a non-zero size means the profile is persisted.

🔒 The connection string is a credential — keep it only in the HF secret. A local copy in neon-db is gitignored so it never reaches this public repo.

Why Chrome for Testing (not regular Chrome)

Branded Google Chrome ≥ 128 silently ignores `--load-extension` (removed by the March 2025 Chrome RFC to curb abuse). On a branded build the extensions never load and chrome://extensions/ shows an empty list with no error in the log.

The fix is to run [Chrome for Testing](https://googlechromelabs.github.io/chrome-for-testing/) — the unbranded build of the same Chrome version where --load-extension still works. The Dockerfile downloads CfT 145.0.7632.117 to /opt/chrome-linux64 and start_hf.sh launches that binary. Everything else (monitor, CDP proxy, flags) is unchanged.

Alternatives if you ever upgrade: use the CDP Extensions.loadUnpacked method (requires --remote-debugging-pipe + --enable-unsafe-extension-debugging), or plain Chromium. Both also keep --load-extension-style loading working.

User-Agent must match the OS

The Space runs on Linux, so Chrome (and the backend HTTP clients) advertise a Linux User-Agent:

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/145.0.0.0 Safari/537.36

A macOS UA on a Linux box is an inconsistency that Cloudflare Turnstile flags, causing the ChatGPT "verify you are human" check to loop forever (the tick never clears). The browser UA lives in start_hf.sh; the ChatGPT backend (free-Chatgpt-api/cookies.go) uses the same UA and Chrome version (145) so the cf_clearance cookie — which Cloudflare binds to the exact UA — stays valid across the browser and the backend.

Endpoints

API gateway

The monitor (monitor/main.go) is also a reverse-proxy gateway: HF exposes only one public port (3001), so all three backend APIs are reached under one base URL by path prefix. The gateway strips the prefix and forwards to the server on localhost:

Auth — required

Every route — the API prefixes, the monitor UI (/), /api/*, /chrome.log, /logs/* — is gated by an `API_KEY` Space secret. Send it as a Bearer token, ?key=, or (for the browser UI) open /?key=YOUR_API_KEY once to set an apikey cookie. Fail-closed: if API_KEY is unset every route returns 503, so the logged-in accounts are never exposed. This Space is also private on Hugging Face, so the URL itself requires HF auth — defense in depth.

1. 1.Set the Space secret `API_KEY` (Settings → Variables and secrets) to a strong random string.
2. 2.Call it:

   curl -X POST https://akash1313-selfapi-v2.hf.space/gpt/api/chat \
     -H "Authorization: Bearer $API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"message":"hello"}'

No/!wrong key → 401. Wrong path → forwarded to the backend (may 404 there).

Project layout

chrome-space/
├── Dockerfile              # builds on akashyadav758/chrome, installs CfT, copies extensions
├── docker-compose.yml      # full self-host stack: browser + 3 API servers on one loopback
├── start_hf.sh             # container entrypoint / Chrome boot sequence
├── profilesync/            # Go tool: Chrome profile <-> Postgres snapshot
├── monitor/                # Go 3-tab live monitor (GPT / Gemini / Flow)
├── free-Chatgpt-api/       # ChatGPT Free API server + gpt-extension
├── free-gemini-api/        # Free Gemini API server + gemini-extension
└── flow-agent/             # Flow automation CLI + Flow-extension

Deploy

This repo is the HF Space — pushing to main triggers an automatic rebuild:

git push origin main

First-time / persistence checklist:

1. 1.Push the code (above).
2. 2.Set the DATABASE_URL secret — see Persistence.
3. 3.Log in once in the monitor UI; the profile auto-backs up within 5 min.

hf-token and neon-db are gitignored — keep your HF token and DB credentials out of commits.

Self-hosting (full stack)

To run the browser and all three backend API servers locally, use docker-compose.yml. The server containers join the chrome container's network namespace (network_mode: "service:chrome"), so they all share one 127.0.0.1 — exactly what the extensions expect. The Chrome profile persists in a named Docker volume (no Postgres needed).

docker compose up --build -d      # browser + chatgpt + gemini + flow
docker compose down               # profile survives in the chrome-profile volume

Or just the browser image:

# Build
docker build --platform linux/amd64 -t chrome-space .

# Run
docker run -d --name chrome-space -p 3001:3001 --platform linux/amd64 chrome-space

# Verify the 3 extensions loaded (look for service_worker targets)
docker exec chrome-space curl -s http://127.0.0.1:9222/json \
  | python3 -c 'import json,sys;[print(t["type"],t["url"]) for t in json.load(sys.stdin)]'

Open <http://localhost:3001> to watch the browser.

Troubleshooting