Menu

What a device tool costs your coding agent: a React Native benchmark
πŸ“°

What a device tool costs your coding agent: a React Native benchmark

DEV Community: mobileΒ·Valentine TikhomirovΒ·5 days ago
#pSwluPNG
#devtomobile#devto#dev#agent#parallax
Reading 0:00
15s threshold

I gave Claude Code five bugs that only show up in a running React Native app. We asked it to find the cause, fix it and check the fix on

the simulator. The app, the bugs, the prompt and the model stayed the same. Only the device tool changed: a bare shell, agent-device,

Argent and Parallax.

These runs used Parallax 0.1.0-beta.3, so the Parallax numbers describe that build. The current version is beta.11.

Every setup fixed every bug. A script checked each fix with real taps on the simulator after the agent had finished, and a separate

model graded the named causes blind. On these bugs, the device tool did not decide whether the agent succeeded. It decided what success

cost.

Parallax costs about what a bare shell costs. With Parallax, a run had a median cost of $0.07, took 36 s and used 205k tokens. With

only a shell (which on this Mac had simctl and AXe), the figures were $0.07, 40 s and 195k tokens.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.