AI agents are getting scary capable. Ask one to run a pentest, chain some automated operations, whatever β it's fast, it's thorough, and when it's done, it hands you a report.
Why should you believe that report?
I treated that question as a threat-modeling problem, not a rhetorical one. Three failure paths, one by one:
1. Logs fail. The log runs in the same process and under the same privileges as the agent. Anything the agent can write, it can rewrite. A process that can edit its own log files produces evidence with zero forensic value for the question "what did this process do?" The basic requirement of evidence is independence from the party being examined. Same-process logging violates it completely.
2. Self-reports fail. A report is a self-description, and self-descriptions can diverge from facts β hallucination, context loss, whatever. LLM output is sampled: ask twice about the same operation and you may get two different reports.β¦