Menu

1 / 5

I asked 3 LLMs to triage bug bounty exploits. So I built a contract that runs them instead

DEV Community: rust·Edy Cu·9 days ago
#Hkwifsnl
#devtorust#devto#dev#exploit#bounty
Reading 0:00
15s threshold

A bug bounty asks the hunter to go first. You hand over the bug, then a human triager decides whether it counts and how much it is worth. The protocol can patch it, call it "known" and pay nothing. And now a growing share of the reports in that queue are AI-written, confident and wrong.

I wanted to see how bad the second problem is, so I measured it. Then I built a bounty where nobody triages at all: the contract runs the exploit and pays the first real break.

Repo: https://github.com/edycutjong/sealed · Live: https://sealed.edycu.dev · Demo (1:34): https://youtu.be/sEMg3q7gG5k

The measurement: can a model be talked into paying?

I built a 52-item corpus against five unmodified OpenZeppelin Ethernaut levels (Fallback, Telephone, Delegation, Force, King): real exploits, near-misses, no-ops, wrong-snapshot runs, malformed input, abuse input, and 15 submissions written by three different LLM families, each with the "hunter's" own explanation attached.

Then I gave every item to three models acting as…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.