Menu

Benchmarking AI Coding Agent Permissions: Manual vs. Auto Mode vs. yolonot
📰

Benchmarking AI Coding Agent Permissions: Manual vs. Auto Mode vs. yolonot

DEV Community: go·Seza Akgün·5 days ago
#WZaP2yS7
#devtogo#devto#dev#yolonot#commands
Reading 0:00
15s threshold

Using CLI coding agents like Claude Code, Codex, OpenCode, or Cursor usually forces you to pick between three flawed options:

Manual prompts: Asks on 100% of commands, which quickly leads to ~93% habitual click-through fatigue where you stop reading the prompt.

--yolo ( --dangerously-skip-permissions ): Misses 100% of risks and gives an LLM raw shell access to your machine.

Built-in harness auto modes: Measured on Claude Code Auto Mode, it still misses 17.0% of risky commands (False Negative Rate), blocks 0.4% of safe commands with no middle ask state, adds LLM latency and API cost to basic read-only commands, and locks your rules to a single tool.

To fix this across all my agents, I built yolonot ( yolonot.dev ), an Apache-2.0 permission firewall written in Go that hooks into 24 AI coding agents .

Benchmark Results (456 Commands Across 3 Open Suites)

We evaluated yolonot across 456 commands from three public benchmark suites ( rogue-security , RedCode-Exec , Dippy CLI, and hidden script attacks) plus…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.