CloudArchitect Lite turns a plain-English description of an app into an AWS architecture diagram, then checks it with 25 deterministic rules. Version 0.3 lets you bring a key from Anthropic, OpenAI or Google. Before I let users pick a model, I wanted numbers, so I benchmarked six models on the same tasks and had a panel of three AI reviewers, one from each provider, score the results blind.

This post covers how the review works, what it found and where the evidence is thin. The full report with every number is in the repository .

The contestants

Two models per provider, matched by price rather than by "flagship" status:

Tier Anthropic OpenAI Google

Economical Claude Haiku 5.5 GPT-6 Luna Gemini 3.5 Flash-Lite

Best quality Claude Sonnet 5.5 GPT-6.1 Sol Gemini 4 Argon

Every model runs through the same pipeline: a requirements step on the provider's economical model, one clarification round, then a design constrained to a catalog of 38 AWS services by structured output.

Step 1: automated checks

Each…