Menu

Making browser perception cheap for LLM agents: snapshots, caching and diffs
📰

Making browser perception cheap for LLM agents: snapshots, caching and diffs

DEV Community: typescript·Haidar Esber·6 days ago
#TdlC9qp0
#devtotypescript#devto#dev#string#snapshot
Reading 0:00
15s threshold

Most "give the model a browser" setups hand it one of two things: a screenshot, or a dump of the full accessibility tree. Both are expensive, and both get re-sent in full after every action.

I've been building ecobrowser , a TypeScript library and MCP server on top of Playwright whose whole job is to make perception cheap for a model. This post walks through three pieces of it, using the real code: a compact snapshot format, a cache keyed on the DOM, and a diff for re-perceiving after an action.

1. A snapshot the model can act on

A snapshot is not a picture of the page. It's a list of interactive elements:

interface SnapshotElement { id : string ; // durable id, e.g. "e3" role : string ; name : string ; tag : string ; value ?: string ; state ?: string []; // disabled, checked, expanded, obscured, ...…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.