Stealth Browser (Camoufox) for pi

Bypasses bot detection / fingerprint blocking that the built-in web_scrape tool hits (Cloudflare, DataDome, 403s, captcha walls). Includes a manual CAPTCHA-solving WebUI and admin log panel.

Deployed: http://192.168.0.65:8010 (login: admin / admin — change it) Health: http://192.168.0.65:8010/health

Architecture

pi (any machine)  --HTTP-->  stealth-browser container (192.168.0.65:8010)
  web_fetch_stealth tool         Camoufox patched Firefox, headless
                                        │
                                        ├─ captcha? → WebUI challenge (login admin/admin)
                                        └─ logs → /api/logs (admin panel tab)

pi extension ~/.pi/agent/extensions/stealth-tools.ts registers web_fetch_stealth, pointing at the server by default. Override with STEALTH_BROWSER_URL, add basic auth with STEALTH_BROWSER_USER / STEALTH_BROWSER_PASS (server uses ADMIN_USER / ADMIN_PASS).

WebUI (admin + captcha solving)

Challenges tab — live queue; when a fetch hits a captcha the job pauses and appears with ⚠. Click Open & solve: live screenshot, click-to-interact, type into focused field, reload/back/scroll, goto URL, then ✔ Done — extract page (or ✖ cancel).
Logs tab — every fetch: start/captcha/action/done/failed/cancelled with status, duration, error. Stored in ./data/logs.jsonl.

Redeploy after changes (on Docker host, 192.168.0.65)

cd ~/stealth-browser       # synced from /blackbox/.../stealth-browser
docker compose up -d --build
curl localhost:8010/health

Proxy (important)

Fingerprint evasion alone gets blocked on datacenter IPs. For Cloudflare/ high-security sites, set a rotating residential proxy:

environment:
  - PROXY=http://user:pass@residential-proxy.example:port

API

GET /health — liveness
POST /fetch — {"url": "...", "extract": "text|html", "selector": "css?", "pause": bool} Blocks until done (or captcha solved via WebUI). Returns {status, title, url, text, outcome, job_id}.
GET /api/jobs — job queue
GET /api/jobs/{id} — job + result text
GET /api/jobs/{id}/screenshot — live JPEG
POST /api/jobs/{id}/action — click|x,y|img_w, type, key, scroll, goto, reload, back
POST /api/jobs/{id}/continue|cancel
GET /api/logs — last 200 log entries

Limits

Defeats fingerprint bot detection. Does NOT defeat:
- IP reputation blocking (needs the residential proxy above)
- CAPTCHAs it can't flag — detection looks for known captcha iframes (Cloudflare, hCaptcha, reCAPTCHA, Turnstile, Arkose); pass "pause": true to always pause for manual review.
MAX_CONCURRENT=3 browser instances; each fetch boots a fresh browser (~2-5s).

Deploy notes / gotchas fixed

Port 8000 was taken by Nextcloud → service runs on host port 8010.
Camoufox needs headless=True explicitly (defaults to headful → DISPLAY error).
Docker blocks Firefox's user-namespace sandbox → MOZ_DISABLE_*_SANDBOX=1 env.
GTK/X11/font system packages required in image.

3.1 KiB Raw Permalink Blame History