Skip to main content

​
What is webrain?

webrain is an open-source (MIT) Rust MCP server for LLM-driven browser automation and web scraping. It is one static binary that drives three browser engines (Chrome, Lightpanda, Obscura) plus a fast no-browser HTTP path, and it exposes 17 intent-based tools over the Model Context Protocol. Install it on any system, point any MCP client (GitHub Copilot, Claude, Codex, Cursor, …) at it, and the model decides everything — search, crawl, scrape, navigate, interact — from a plain-language prompt. No hardcoded intent detection, no daemon, no Node.js.

​
Why webrain?

Web automation shouldn’t mean wiring up a driver, a browser download, and your own tool wrappers before an LLM can touch a page. Webrain collapses that into one binary + one install command and speaks MCP, so any LLM client drives it directly.

Quickstart

Get a browser engine, start the MCP server, and drive it from a prompt in minutes.

​
What you can do with it

  • Scrape at scale — webrain_batch pagination + webrain_crawl (spider) with auto-throttle and checkpoint/resume; webrain_crawl (sitemap / scan) to map a site first.
  • Structured data without hand-written selectors — webrain_extract (autoschema) probes the DOM, then schema / regex / table modes read container-level structure.
  • Stealth login — real-Chrome profiles with an encrypted local credential vault (AES-256-GCM + optional TOTP); transfer cookies across engines.
  • Get past challenges — reads the challenge field on every navigate and clears Cloudflare/Turnstile via real Chrome + a persistent profile and native login.
  • See the page — webrain_observe (a11y / semantic / state), plus vision tiles (screenshot → vector store) for tables and charts.
  • Read anything — webrain_pdf (extract + render), JSON-LD, media, plus webrain_scrape for static pages 10–100× faster than a browser.

​
How it’s built

Everything speaks Chrome DevTools Protocol over a raw WebSocket, so one backend drives Chrome, Lightpanda, and Obscura interchangeably.

​
Prerequisites

  • OS: Windows, macOS, or Linux (x86_64 / arm64).
  • A browser engine — run webrain install once (downloads Chrome for Testing). Obscura and Lightpanda are optional extra engines.
  • Linux: system libraries for Chrome — see Installation.
  • Docker — only to run the obscura / lightpanda engines in containers.
  • An MCP-capable client (VS Code + Copilot, Claude, Codex, Cursor, …) — optional; the CLI works standalone.

​
Where to go next