A tool-calling agent that reads a policy correctly can still apply it wrongly. In my test, Gemini 4 Carbon fetched a 30-day return policy, then refunded an order delivered 46 days earlier, three runs out of three. Qwen 4 Max refused it three out of three. This post shows the test, the grader, and the validation that makes the question matter much less.
Disclosure: I ran this on apimodels.app, which is the API I work on. apimodels.app is a multi-model API gateway: one key and one OpenAI-compatible endpoint for image, video, audio and language models, including both models here. I wrote this from my own test data, with AI assistance for drafting and editing.
The task
The customer says the space bar on their keyboard is broken and asks for a refund.…