Skip to main content

​
Tools Reference

All 18 tools are discovered dynamically — webrain_guide lists them for the LLM. The surface is intent-based (firecrawl-style): each tool has a what / action / op / mode selector that routes to the legacy capability via map_surface(). This page is the authoritative reference; descriptions match webrain-mcp/src/tools.rs. Eleven page tools declare an optional session_id (navigate, observe, interact, eval, eval_in_frame, extract, vision, pdf, crawl, search, serp) that routes the call to a webrain_open_session pool. The browserless tools do not declare it, and webrain_batch spreads across browsers with cdp_urls instead.
The accordions below document the legacy one-action tool names (e.g. webrain_click, webrain_snapshot) that still dispatch — they are the executors map_surface() routes the consolidated selectors to, so they remain the precise reference for each capability.

​
Tool-to-code map

The consolidated schemas live in webrain-mcp/src/tools.rs (list_tools), and every call routes through call_tool — map_surface(name, args) first folds a consolidated tool + selector into the legacy arm name, then the existing match runs. From there tools resolve into webrain-core. The execution model is simple: browser tools run through CdpBackend (one shared WebSocket to the engine); a handful of tools skip the browser entirely. Shared vs engine-specific: everything behind CdpBackend works on Chrome, lightpanda, and obscura alike — the differences are capability (screenshots, challenge solving, a11y fidelity), not protocol. Tools with no browser dependency (webrain_scrape, webrain_search, webrain_serp HTTP engines, webrain_crawl mode=sitemap, webrain_download, webrain_pdf, webrain_watch) are intercepted in webrain-mcp/src/lib.rs handle_rpc before a backend is even attached.

​
Host

Liveness for the machine webrain is running on, read straight from /proc (no shell, no sidecar):
  • load — 1/5/15-minute load averages, so a slow job can be told apart from a busy host.
  • memory — total_mb and available_mb. Available, not free: MemFree counts page cache as gone, which reads as an emergency on a healthy box.
  • processes — total, plus browser_processes counting engines by comm (chrome, chromium, headless_shell, obscura, obscura-worker, lightpanda). One Chrome shows up as several (zygote, GPU, renderers), so treat it as a liveness signal, not a tab count.
  • uptime_s — how long the container or host has been up.
available: false means the platform has no /proc (macOS, Windows). That is a fact to report, not a number to invent.Takes no arguments and is read-only. Disk usage is deliberately absent: every way to get it costs either a dependency or a child process, and spawning processes is the thing this tool exists to avoid.
Agent decision guide: browser selection (real Chrome vs obscura vs lightpanda vs fetch_http), how to handle Cloudflare/CAPTCHA/Turnstile challenges (check the challenge field after webrain_navigate; persistent profile + real Chrome + session via webrain_session(op=login)), and the extraction tool matrix. Call FIRST when unsure which webrain tool/browser to use.It answers progressively: with no topic you get the tool list plus an index of every section; topic="extraction" (or any section name — case-insensitive substring) returns just that section; topic="all" returns the whole guide. That keeps the “I’m not sure what to do” call at ~800 tokens instead of ~2.7K.topic="skills" lists operator skill files from WEBRAIN_SKILLS_DIR (default <engine cache>/skills), and topic="skill:<name>" returns one — a way to hand the agent house rules (a customer’s conventions, a login procedure) without recompiling. Names are validated (a-z A-Z 0-9 _ -, max 64) so the argument can never carry a path.Two ways to fill that directory:
webrain skills add requires a frontmatter name: and refuses files over 64 KiB — both are the reader’s own restrictions, so a skill that installs is a skill that can be read.
Navigate to a URL and return page state (title, visible text, interactive elements, deduped same-origin links) plus a challenge field (cloudflare_challenge | blocked | captcha) when the page is gated. Use links for one-call crawl / internal-link discovery. If challenge is set, see the Agent Decision Guide for the real-Chrome bypass.Optional request-quality params: disable_resources (block fonts/images/media), network_idle, wait_selector + wait_selector_state, css_selector (narrow returned text), block_trackers.
Re-capture current page state WITHOUT navigating. D1 DOM-fingerprint skip: returns the cached state unchanged when the page hasn’t mutated, saving tokens.
Run arbitrary JavaScript in the current page and return the JSON result. Use for precise structured extraction (e.g. product schemas).
Just-in-time page context: viewport/page size, scroll position, pixels/pages above & below, position %. Tells you when to scroll before interacting. No DOM dump.
Accessibility-tree snapshot: [{role, name, value, css_path}]. Read-only — understand structure, then interact via webrain_navigate/webrain_snapshot element indices. Filters: role, filter (substring on name OR value OR css_path), max_nodes.ARIA cheat-sheet: combobox (dropdown), option (menu item), tab, radio (segmented control), textbox, button. If role=<x> returns [], drop the role filter and filter on the label text.
Semantic-tree text snapshot of the current page: role "name" lines for the LLM, plus the raw AX JSON.
LAST RESORT — full raw HTML (token-heavy, unreadable). Never use for page text: webrain_snapshot/clean/eval/extract_json return text/structure cheaper. Only when the task explicitly asks for HTML markup.
Return captured page errors/warnings (uncaught errors + unhandled rejections) since the last call. Injects a listener on first call.
Take a screenshot of the current page. full_page captures the full scrollable page.format: png (default) | jpeg | webp | avif, plus quality (1-100, default 80) for the lossy ones. png/jpeg/webp are encoded by the browser itself (Page.captureScreenshot), so those cost no extra pass. avif has no CDP encoder: it is captured as png and converted in-process by ravif/rav1e, which costs a CPU re-encode.Measured, same viewport, two very different pages:Two lessons in that table: jpeg can be larger than png on a flat page (JPEG pays a per-8x8-block cost even for solid white, while deflate sees runs of identical pixels), and avif at speed 6 is not smaller than webp — its advantage needs a much slower encode. Use webp unless a client needs a specific type. The reply reports format (what was actually produced) and requested_format, so a silent fallback is visible.The reply also lands the image as an MCP image content block next to the JSON, so a client that renders images shows it directly instead of handing the model a base64 string. The same bytes are written to dir and the path is returned.
Requires a paint engine — real Chrome, or Obscura v0.2.0+ render builds (default install). No-render builds and lightpanda have no paint engine (lightpanda returns a fake placeholder PNG).
Annotated viewport screenshot: overlays numbered red boxes on interactive elements and returns a legend [{n, index, tag, text}]. The index maps to webrain_click/webrain_type indices. Built for vision models — read the labels, then click by index. Removes the overlay after capture.

​
Interact

Click an interactive element by its index (from webrain_navigate elements list).
Type text into an input element by its index.
Add ms to type through real per-key Input.dispatchKeyEvent events at that cadence instead of one Input.insertText paste. insertText fires no keydown/keyup, which text captchas and the reCAPTCHA/hCaptcha audio challenge flag as non-human — {"index": 5, "text": "hello", "ms": 60} produces 5 keydowns.
Press a key in the focused element. Use after webrain_type to submit forms.Keys: Enter Tab Escape Backspace Delete Space ArrowUp/Down/Left/Right Home End PageUp PageDown Insert.key also accepts a shortcut — ctrl+a, shift+Tab, meta+v, alt+ArrowLeft. The modifier bitmask rides on both keyDown and keyUp and the base key carries a real code (KeyA), which is what makes an accelerator bind and what clears a prefilled captcha field or tabs through an audio challenge. Enter keeps the JS path that dispatches form.submit(); a modified Enter does not submit.
Scroll the page up or down. direction: up | down.With x, y and an optional delta it wheels the element under that point instead of the window — the only way to move a captcha modal, an inner list or a scrollable side panel. delta is signed (negative scrolls up); default ±500.
Browser navigation: go back, forward, or reload.
Manage browser tabs. new (url, returns id, becomes active) | switch (id) | close (id) | list.
Remove visible fixed/sticky overlays (cookie banners, popups, modals) that block interaction.
Select an option in a native <select> dropdown by index: matches by option value OR visible text, fires a real change event. No-match errors and lists the available options so the LLM self-corrects.
Hover an element by index via trusted CDP mouseMoved. Triggers CSS :hover menus, tooltips, and lazy hover-reveal content.
Set a checkbox/radio to a state by index: trusted click, verifies, falls back to JS label-retarget. Returns the ACTUAL checked state so the LLM can verify.
Resolve a pending JavaScript dialog (alert/confirm/prompt). A sync alert() pauses the page — every click/eval hangs until this resolves it. action: accept | dismiss (optional prompt_text). Works even while the renderer is paused.
Standalone wait after an action: wait a fixed ms, or poll until a selector or visible-text substring appears (default timeout 15s). navigate already waits internally — this is for click→AJAX→render steps. Returns satisfied: bool.
Upload files to a file input by index via CDP DOM.setFileInputFiles. Index must be an <input type=file>.
Trusted click at raw viewport coordinates. For cross-origin iframe content and reCAPTCHA checkboxes where JS clicks only focus the element. Get coords from a screenshot or webrain_page_info.
Trusted drag across viewport coordinates (x1,y1 → x2,y2): approach the handle, press, dwell, trace an eased path with the button held, settle on the target, release — all via CDP Input events. Built for slider/drag CAPTCHAs (“slide to unlock”, GeeTest, DataDome, AWS WAF) and it works across cross-origin iframes where synthetic JS events are ignored. Get coords from a screenshot (webrain_observe what=screenshot) or webrain_page_info.ms sets how long the handle is held before the movement starts (default 150). A zero-length drag (x1==x2, y1==y2) with ms is therefore a press-and-hold in place.
Trusted press-and-hold at (x, y) for ms, then release — for “Press & Hold to confirm you are a human” widgets (PerimeterX/HUMAN). Those only fire after a multi-second mousedown, so a click or a short drag never registers. Default 6 s.action=hold, or the equivalent zero-length drag (x1==x2, y1==y2, ms).
Register a JS init script that runs before EVERY future navigation (Page.addScriptToEvaluateOnNewDocument). Use for closed-shadow-root piercing, API stubs, or route/UA overrides. Accumulates for the session.

​
Extract

Detect repeated container patterns on the page. Returns candidate base-selectors with occurrence counts for building a webrain_extract_json schema. Zero LLM. min_occurrences (default 3).
CSS-schema extraction: build a JSON array from a base selector + field selectors. Zero-LLM structured extraction.
type: text | attr | html | xpath. adaptive: true auto-relocates the container when the base selector matches nothing (site redesigned).
Regex pattern extraction over the current page: built-ins email/url/phone/price/date/time/ip/uuid + custom [{label, re}]. Scans page HTML (catches href/mailto).
Extract all HTML tables on the current page to JSON. Returns arrays of {header: cell} row objects per <table>. Zero LLM.
Extract JSON-LD / microdata from the current page. Returns parsed <script type="application/ld+json"> blocks — schema.org product/article/organization data. Zero LLM, zero cost.
BM25 relevance filter: score a list of text items against a query, keep the top-k. Zero LLM. Use after extraction to keep only relevant results.
Clean page text: strip nav/footer/script/style/iframe, exclude social/ads links, filter by word length. Returns clean text blob (max 8KB). In-page JS, zero-LLM.
Prune the current page to its dense content (crawl4ai PruningContentFilter): strips nav/footer/aside/form/header boilerplate and returns the meaty text, scored by text-vs-link density + tag importance. Use instead of raw innerText — fewer tokens, denser signal.
Full composed page text including Shadow DOM (crawl4ai flatten_shadow_dom). Web-component sites (Lit/Stencil/Shoelace) render content in shadow roots that querySelectorAll/innerText miss. Resolves slots, recurses open shadow roots. Use when a page looks empty.

​
Crawl & Batch

Batch over many URLs using concurrent tabs. op: fetch (read text) | markdown (full page as Markdown) | extract (CSS schema: base_selector+fields) | autoschema (propose base selectors per URL — the step that produces a base_selector for extract, same in-page heuristic as webrain_extract{mode:"autoschema"}, min_occurrences default 3) | interact (async JS interaction, then optional extract) | eval (run JS in each tab) | screenshot (save PNGs to dir).
cdp_urls fans out across N CDP backends round-robin (per-proxy isolation). output persists the full payload to disk. Optional disable_resources, network_idle, wait_selector shared with navigate.
Crawl a website from a seed URL (BFS / DFS / BestFirst). allow/deny URL-regex filters, retry, delay_ms, autothrottle (adaptive backoff), crawldir checkpoint/resume (persists {queue, seen, results} atomically; resume returns prior + new, never re-fetches; max_pages counts the whole crawldir), concurrency (parallel tabs on Chrome/obscura, default 4), crawl_timeout_secs.
Discover crawlable URLs from a site’s sitemap. Follows robots.txt Sitemap: → index → leaf sitemaps → every <loc>. Pure HTTP, no browser. Returns {urls, count, sources}.
Auto-scroll the page to trigger infinite-scroll / load-more content. Returns {scrolls, height}. Run before extraction on SPA feeds.
Validate a list of URLs — alive vs dead. Filters 404s/5xx/errors. HEAD first, GET fallback. Use before batch extraction.
No-browser HTTP fetch: GET a URL, return {url, status, text}. 10–100× faster than browser navigation, zero memory — but no JS/SPA/auth. Use for static pages.
Search the web and navigate to the results page. duckduckgo is HTML-lite and scrape-friendly (default); google and bing return plain HTML via HTTP; brave returns an SPA shell — use webrain_navigate to Brave’s URL instead for real results.
Structured search results as typed JSON (position/title/url/domain/snippet) — a SERP API any LLM can call. duckduckgo (default) / bing are plain HTML over the pooled no-browser HTTP agent; google is JS-gated over HTTP so it uses the browser path — webrain serp --engine google auto-launches a persistent-profile Chrome (headed by default, --headless for headless) and drives homepage → consent (trusted two-phase click) → humanized typing → trusted click; brave uses its keyed API when BRAVE_API_KEY is set (no browser needed — 20/page, page-indexed offset), otherwise it renders in the connected CDP engine; auto fetches HTTP engines concurrently and merges + dedupes.Built-in: provider fallback (fallback), URL dedupe, pagination (page), safe (safe) + region (region — an en-US market is pinned when unset, because a GeoIP’d IP otherwise serves localized garbage), request_id + ms, retry with backoff (retries), and a multi-page merge so limit > ~10 is honored wherever engines paginate.Proxy: proxy routes the HTTP engines through an HTTP(S)/SOCKS proxy (e.g. http://user:pass@host:port) and bakes --proxy-server into the google auto-launch. Bing/Google GeoIP-lock flagged/rotating IPs and ignore market params, so a clean-IP proxy is the reliable way to get correct, larger result sets. An attached CDP engine keeps whatever proxy it started with.CAPTCHA walls: detected, never solved automatically. webrain_navigate reports challenge (cloudflare_challenge · captcha · blocked) so the agent stops instead of extracting a challenge page; passing one needs real Chrome + a persistent profile + an interactive webrain login.Brave API: set BRAVE_API_KEY and engine=brave goes through api.search.brave.com — typed JSON, 20 results per page, no browser at all; auto engages it for limit > 20 while still short, before any browser render.serpapi.com: set the standard SERPAPI_API_KEY env var. It’s the reliable way to get MORE than the free engines’ ~10-result cap — serpapi honors num up to 100 — so for limit > 10 it’s tried first for google (the full best result set); for limit <= 10 the free browser path stays primary and serpapi is a fallback. Unset key / quota-exhausted / 4xx degrades to the normal fallback with no error.
Also on the CLI: webrain serp "query" [--engine …] [--limit N] [--page N] [--safe] [--region R] [--no-fallback] [--json] [--headless] [--proxy URL] [--fresh] [--pipe] [--stealth] [--hold].

​
Proxy routing — three places

An explicit proxy always wins over WEBRAIN_PROXIES; the pool is only the fallback, and it is process-global — never share it between tenants. A browser tab’s exit is fixed when its browser or context is created, so “change proxy on a live browser” means re-opening the session_id with a different proxy (new context + tab, same Chrome process) — not mutating the existing one. A new context is a fresh profile: keep_identity:true copies cookies only, so an SPA that keeps its token in localStorage/IndexedDB still logs out. Where an IP change is a fraud signal (banks, Google, Stripe), pin one exit per profile and relaunch with the same <service>/<profile> — the on-disk profile keeps the identity, and --proxy-server is the guaranteed path on every engine.Default (google) = warm persistent profile + session — the real bypass path (see skills/webrain/references/challenges.md): the auto-launched Chrome stays alive on 9222 between runs, accumulating consent/session cookies until it becomes a trusted Google profile. --fresh opts OUT of the warm session: it always launches a brand-new profile + cookies on a free port, so the consent modal always renders and is always dismissed before the humanized flow — use it when you want deterministic consent (e.g. a suspect/stale profile). --hold keeps the launched Chrome open after the search so you can watch it (press Enter to close). --pipe (with --fresh) launches that Chrome via --remote-debugging-pipe (NO open debugging port — the open port is the automation fingerprint Google walls on /sorry; note Chrome’s pipe CDP is broken on Windows, works on Linux/macOS). --stealth opts IN to the launch-flag stealth (--disable-blink-features=AutomationControlled); the default is a plain launch + CDP-level masking (patchright/browsemind’s recommended combo). All google human-like actions are trusted CDP input (Input.dispatchMouseEvent/Input.dispatchKeyEvent), never JS-driven.

​
Vision

PixelRAG-style tile capture: split the current page into a grid of screenshot tiles (base64 PNGs) so a vision model can read regions — tables/charts/layout survive. tile_width/tile_height (800), max_tiles (16).
Capture the current page as vision tiles, embed each via EMBED_URL (OpenAI-compatible /embeddings), and add to a cosine index persisted to vision/{tag}.jsonl. With the bundled local vision model (webrain install vision), the response also includes vision — a Qwen3-VL-2B page caption via llama-server (real understanding).
Embed a text query and return the cosine top-k stored tile ids from a vision index — semantic page retrieval. tag, query, k (5).
Screenshot a viewport / clip region and ask a vision LLM an arbitrary prompt (captchas, visual QA). Provider chain is cloud-first: OPENROUTER_API_KEY (Qwen3.6-27B) → OPENAI_API_KEY → FIREWORKS_API_KEY → GROQ_API_KEY → bundled local Qwen3-VL-2B (webrain install vision) — a 27B cloud model reads captcha tiles the local 2B misreads, and a flaky provider fails over to the next.
  • clip x,y,w,h (default: full viewport), optional scale (2-3× upscale for small tiles; the answer’s coords divide by scale to get viewport px).
  • tiles: array of {x,y,w,h} clips sent in ONE request as numbered images — the batched precision pass for grid captchas (1 call, not N). Keep scale:2 on cloud models; scale:3 only helps the weak local 2B.
  • Note: Qwen3 dumps a long <think> that truncates multi-image answers in tool results and misreads distorted text — for text/assemble puzzles read each tile as its OWN single-image op=ask.
Run JS inside a specific cross-origin iframe (matched by url_contains src substring) via a CDP isolated world — the only way to read exact geometry inside reCAPTCHA/hCaptcha/Turnstile/2captcha challenge frames (grid tile rects, verify button, shared sprite data-URLs) that webrain_eval cannot reach. Returns the expression’s JSON value (string results need parsing). Same-origin iframes can be reached with plain webrain_eval via contentDocument. See workflows/captcha-solve.md for the exact puzzle-geometry expressions.
Watch any video (URL or local file): timestamped transcript (yt-dlp captions → local whisper-cli → cloud Whisper API) + frame file paths, no browser needed. detail transcript/efficient/balanced; batch sources[] in parallel. vision:true sends up to 3 sampled frames to a vision LLM (Groq qwen3.6-27b → OpenAI gpt-4o-mini → local Qwen3-VL-2B when no key) and returns text captions + a fused visual summary in vision. First run: webrain install watch (mono bundle) + webrain install vision for the local hero.

​
PDF & Media

Save the current page as PDF (base64-encoded).
Convert a PDF to Markdown (pure Rust on lopdf). Returns page count, pdf_type (TextBased/Scanned/Mixed), confidence, layout (is_complex, pages with tables/columns), full markdown, and per-page texts. Proper ToUnicode CMap decoding fixes LaTeX/CID-font PDFs. Single path or batch paths.
Render PDF pages as base64 PNGs — the vision-model alternative to text extraction (bypasses font encoding issues). Optional tile_size splits each page into square tiles; every entry carries page, x, y, w, h. Requires a --features pdfium build and pdfium.dll / libpdfium.so on PATH. Prebuilt binaries ship without pdfium, so op=render errors there — use op=extract for text and tables, op=images for embedded figures, or open the PDF in Chrome and webrain_observe what=screenshot for a raster.
Extract embedded images/figures from a PDF as base64 PNGs — zero system deps. Handles DCTDecode (JPEG) and FlateDecode. Skips JPEG2000/CCITT/JBIG2 (use webrain_pdf_render for those). Works in the default build.
Download single or many files/video/audio. engine http (plain streaming, optional filter_extension) or ytdlp (HLS/DASH/.m3u8, playlists, age/cookie-bound media — full yt-dlp passthrough via args).
Discover media URLs the page loads. With a url: CDP Network capture of the full load — catches JS-loaded .m3u8/.mp4/manifest/player-API requests. Without: Performance API + <video>/<audio>/<source> scan of the current page.
List images on the current page: [{src, alt, width, height}]. Useful for product/photo URLs.

​
Auth & Sessions

Fully-automatic login from the local vault (or WEBRAIN_USER/WEBRAIN_PASS): the server decrypts the secret in-process and injects it via CDP — the value never passes through the model. Auto-discovers login fields; on a 2FA gate it TOTP-injects if a seed is stored and returns waiting_for_human: true. Reply is status-only.
List vault profiles (service, profile, username, created_at) — names only, never secrets.
Read all cookies (incl. HttpOnly) from the session backend. Use with webrain_setcookies for cross-browser session migration.
Import cookies into the session backend for cross-browser auth. MUST be followed by webrain_batch WITHOUT cdp_urls so set + batch share one connection.
Export the current browser’s auth state (cookies + localStorage) to <profiles_dir>/<service>/<profile>/state.json so a login follows you across machines.
Import auth state from <profiles_dir>/<service>/<profile>/state.json into the current browser. Navigate to the target site first — localStorage is origin-scoped.
Create a named browser session pool. Direct webrain_batch/webrain_navigate calls to it via session_id — use different sessions to isolate tasks or browsers (e.g. one per CDP_URL for parallel subagents).
Destroy a named session pool and its browser backend. The default session cannot be closed.
List all active session pools with their session IDs and CDP URLs — discover sessions for subagent routing.
Stop a Chrome launched by webrain launch (kills the browser process; the persistent profile + cookies remain for the next launch).