Headless Chrome is useful.

It is also overused.

For years, the default answer to “this page is hard to scrape” has been some version of:

Use Puppeteer. Use Playwright. Add stealth. Wait for the page. Extract the DOM.

Enter fullscreen mode

Exit fullscreen mode

That works often enough that it became muscle memory. But using a browser as the first step for every page is expensive, slow, operationally annoying, and frequently unnecessary.

I’m building webclaw , a web extraction API, CLI, and MCP server for AI agents. One of the biggest architecture decisions was this:

Do not make the browser the default path.

Enter fullscreen mode

Exit fullscreen mode

The browser is an escalation path.…