Most phone automation needs a computer on a cable, a developer account, or a jailbreak. ESP32 iPhone Agent takes a different route: a small ESP32-S3 board plugs into the iPhone and acts as the agent's eyes and hands, while an iPhone app runs the vision model loop. At run time it is just the phone and the board.

This guide walks through the setup from the project's README, start to finish, so you can type a goal like "Open Settings and go to Wi-Fi" and watch a vision model do it on a real iPhone.

How it works in one paragraph

The board opens its own Wi-Fi hotspot and shows up in the iPhone's Screen Mirroring list as ESP32 Agent . The phone mirrors its screen to the board over AirPlay, and the board forwards that stream to the app over Bluetooth LE. The app sends screenshots to any OpenAI-compatible vision model, which plans the next action. The board is also a standard USB mouse and keyboard, so with AssistiveTouch on, a click is a tap and a drag is a swipe.…