Webrain turns an unknown page into structured JSON without you writing a single
selector. The zero-LLM discovery pipeline is:
1. Discover the container
webrain_extract (mode: autoschema) detects repeated container patterns on
the page and returns candidate base-selectors — e.g. .product, li.item.
Zero LLM.
2. Probe the structure
Run small JavaScript to inspect what lives inside a container before you
declare fields.
webrain_extract (mode: schema) builds a JSON array from a base selector +
field selectors:
Set adaptive: true to auto-relocate the container when the base selector
matches nothing (site redesigned) — it finds elements still containing ≥2 of
the field selectors.
Token discipline
Never reach for webrain_observe (what: html) for page text.
webrain_observe (what: state / clean), webrain_eval, and the
extractors return text/structure far cheaper. html is the LAST resort —
only when the task explicitly asks for raw markup.
The full recipe for paginated catalogs
Proven flow — no class assumptions, no get_html.