Headless Chrome is useful.

It is also where a lot of scraping systems go to become slow, expensive, and impossible to reason about.

I am building webclaw , a web extraction API, CLI, and MCP server for agents and LLM apps. One architecture decision keeps paying for itself:

The browser is a fallback. Not the default.

Enter fullscreen mode

Exit fullscreen mode

That sounds obvious until a target page gets annoying.

Then the reflex kicks in:

Use Playwright. Launch Chrome. Wait for network idle. Extract the DOM. Ship it.

Enter fullscreen mode

Exit fullscreen mode

It works often enough that people confuse it with a strategy.

It is not a strategy.

It is an expensive hammer.

Browser-first scraping demos well

The browser-first pipeline is simple:

URL -> browser -> rendered DOM -> extraction

Enter fullscreen mode

Exit fullscreen mode

For demos, this is great.

You point Puppeteer or Playwright at a page. JavaScript runs. The DOM appears. You grab text.…