Browser automation for AI agents.
One Rust binary that gives any LLM a real browser. Scrape, clear bot gates, and transcribe videos, all on your machine.
curl -fsSL https://raw.githubusercontent.com/prokopis3/webrain/main/scripts/install.sh | bashNo Node, no Python, no Docker. Runs on macOS, Linux, and Windows.
webrain mcpnavigate shop.example.comobserve product cardsextract jsonvault add store.example.comlaunch chromenavigate auth.store.example.comsession login service=store profile=defaultbatch extract · 12 urls (session)batch interact · 40 urls in parallelwebrain mcpnavigate shop.example.comobserve product cardsextract jsonwebrain mcpwebrain_navigate shop.example.comwebrain_extract mode=autoschema
Give the scraper LLM a try.
This is the agent loop webrain runs: a plain-language prompt, seventeen intent-based tools, and structured JSON back. Presets show illustrative output, except the drone one: a real 1m43s run. Pick a demo or type your own prompt.
webrain_navigate→ ok · 0 challengeswebrain_observe · what=state→ 24 product cards detectedwebrain_extract · mode=autoschema→ schemawebrain_extract · mode=schema→ 42 rowsFrom install to scraping in three steps.
No selectors to hand-write, no daemon, no driver wiring. The binary, your client, and the model do the rest.
One binary, any OS
Install webrain once. No Node, no Python, no Docker. Engines download on demand.
curl -fsSL …/install.sh | bashRegister it in a client
Add two lines to Claude, Cursor, Copilot, or Codex. The client spawns webrain mcp.
Point the model at the web
Tell the agent what to do in plain language. It navigates, extracts, and returns structured data.
scrape product prices from URL XOne loop, every site.
Read the page, pick a tool, act, hand back structured JSON. The agent walks the same circuit on every job.
Everything an agent needs to read the web.
Seventeen intent-based tools cover the full job. No selectors to hand-write, no driver plumbing, no daemon.
Structured extraction
Autoschema probes the DOM, then schema, regex, table, JSON-LD, and BM25 modes return clean data. Zero LLM, zero cost. Live: 407 products from a real store in ~17 tool calls.
Stealth login
Real-Chrome profiles with an encrypted local vault and optional TOTP. Cookies transfer across engines.
Challenge bypass
Every navigate reports a challenge field; real Chrome with a persistent profile and native login clears Cloudflare, Turnstile, and reCAPTCHA gates. No Python sidecar.
Crawl at scale
Batch, spider, sitemap, and scan modes with auto-throttle and checkpoint/resume. 132 products in one parallel call, ~2-4 s/page on Lightpanda.
Read anything
PDF extract and render, video transcripts plus frames, downloads, JSON-LD, media, and a no-browser HTTP path. Video transcripts run fully offline with bundled whisper and local vision.
Local AI, all OS
Bundled whisper-cli for transcripts and Qwen3-VL-2B for vision. No cloud key required. Runs fully on your machine.
Skills, recipes, and agent prompts, in the repo.
webrain ships a working agent playbook, not just tools: a routing skill, verified block-bypass workflows, and a ready-to-copy news-research prompt. Clone the repo and load them into any client.
webrain skill
The router. Load it and it picks the browser, the challenge path, and the extractor for each task, mcp_webrain-* without guesswork.
Workflows
Verified block-bypass recipes: CAPTCHA geometry, feed & news aggregation past WAFs, protected-site end-to-end.
Agent prompts
A ready news-research agent: copy the prompt into any chat for the optimal tool order and a precise bypass playbook.
References
Depth for every decision: browser selection, challenge handling, persistent profiles, extraction matrix, anti-patterns.
Proven on real jobs.
Measured on live sites and documented in the repo. No invented numbers, no benchmark theater.
Reproduce the zero-LLM claim on any site: webrain_extract · mode=autoschema → mode=schema. No LLM, no tokens.
Real operations, end to end. Two full agent jobs. Every number from the run, not a demo.
Buy a machine for Qwen3.8
Live skroutz.gr op: navigate 4 targets, batch 2 search pages, eval the benchmark tables, rank by €/tok/s.
Verdict → Acer Predator Helios 16 AI · 3.699 €, ~100 t/s on Qwen3.8-27B Q4 (interpolated)
Plan a drone build
Multi-source research: serp discovery → one batch over 6 build guides + firmware repos → a synthesized parts-and-code plan.
Output → 12-part list · $500-650, Betaflight / INAV / ArduPilot + STM32 KiCad repo
Every figure is in the repo: the agent guide, the changelog
Structured search. Five engines, one tool.
Ask for results and get typed JSON: position, title, url, domain, snippet. duckduckgo and bing answer over plain HTTP with no browser at all; google and brave render in the connected engine; auto fetches duckduckgo + bing in parallel over plain HTTP and merges them — relevance-filtered and deduped (google and brave join the merge when a CDP engine is attached).
typed resultsposition, title, url, domain, snippetfive engines + autoduckduckgo, bing, google, brave, autofallback + dedupeproviders fall back, duplicate URLs droppedpagination + safe + regionpage offsets, safe search, en-US market defaultproxy + captcha solveroute engines through HTTP(S)/SOCKS proxies; 2captcha /sorry auto-solverequest_id + retrycorrelate every call, retry with backoffOne MCP tool and one CLI command: webrain_serp and webrain serp.
webrain_serp · engine=duckduckgoOne CDP backend, four ways to load a page.
Pick the right engine for the job. From a full Chrome for interactive SPAs to a no-browser HTTP fetch for static pages.
Chrome
Interactive SPAs, challenges, screenshots. Anything real Chrome can do.
Obscura
Fast batch scraping of non-challenged JS pages. No paint engine, no screenshots.
Lightpanda
Real accessibility tree, lightweight. Screenshot returns a placeholder frame.
fetch_http
Static HTML, no JS, no auth. 10-100× faster. No engine at all.
Point any LLM at the web.
No driver wiring, no browser download dance, no tool wrappers. Install once, and your agent drives a real browser.
stdio or HTTP. Works with Claude, Cursor, Copilot, Codex, Gemini.
Everything runs on your machine. No cloud, no API key, no data leaves.
Challenge bypass, encrypted vault, TOTP, and cookie transfer across engines.
Say it plainly. The agent picks the tool.
No selectors, no wrappers. The model reads the page, reads the challenge field, and dispatches to the right tool. These are real pairings from the agent guide.
How it stacks up against the alternatives.
Honest, feature-by-feature comparisons against the tools you may already be using.
Why an MCP-native tool beats a driver library for LLM agents.
Local-first and free versus a hosted API. Zero cloud, zero keys.
One binary and real CDP engines instead of a Python agent loop.
Rust-native extraction strategies and vision with no Python runtime.
Questions, answered plainly.
The objections that come up before you install. The short version: yes, it runs on your machine.
Do I need Node, Python, or Docker?
No. webrain is a single Rust binary. The browsers it drives (Chrome for Testing, Obscura, Lightpanda) download on demand into a cache dir. Nothing else to install.
Does it cost money or need an API key?
No. It is open source (MIT) and local-first. Cloud keys are optional and only for video transcriptions and vision; the bundled local whisper and Qwen3-VL-2B models work offline.
Which AI clients does it work with?
Any MCP client: Claude, Claude Code, Cursor, GitHub Copilot, Codex, Gemini, and more. Register it once in the client’s MCP config and the model drives the browser itself.
How is this different from a scraping API?
webrain runs on your machine, sends no data to a third party, and gives the LLM a real browser with stealth and challenge bypass. For static pages it can be 10-100× faster than a full browser.
Give your agent a browser in five minutes.
Install webrain, start the MCP server, and ask your LLM to scrape. No selectors, no wrappers.