Building an AI code reviewer is one thing. Trusting it with real pull requests is another.

I've been developing CodeGuard AI, a self-hosted AI-powered GitHub PR reviewer built with Java 21 and Spring Boot 3.

Instead of relying only on sample code and happy-path demonstrations, I've been testing it against deliberately created GitHub pull requests with different bugs, test scenarios, and edge cases.

What I've been testing

So far, the test lab has covered scenarios across several areas:

Test execution and validation

Passing test suites

Genuine assertion failures

Test execution timeouts

Builds that succeed without executing tests

Unreadable test-result evidence

Bug detection

Unsafe Optional.get() usage

Null input handling

Incorrect discount calculations

Boundary-value errors

Incorrect implementations hidden behind passing tests

False-positive resistance

Safe Optional.orElse() usage

Correct discount calculations

Valid test assertions

Harmless code that should not trigger unnecessary…