Your client timeout is a promise about someone else's server. Most teams never check whether that promise is true — they copy timeout=30, retries=2 from a tutorial, ship it, and discover the real distribution during an incident.

This post is a way to stop guessing. You will measure a free-tier model endpoint yourself, turn the samples into a decision table, and block promotion when the evidence is too thin to support a number. The artifact is a dependency-free probe script plus a checklist you can paste into a pull request.

The number nobody measures

Latency percentiles on a shared endpoint are not a property of the model. They are a property of the model, the queue in front of it, your region, and the hour of the day.

That means an average latency figure from a vendor blog post tells you almost nothing about your own traffic path. You need your own samples, taken from the place your code actually runs, with the prompt sizes you actually send.

And you need the failure side, not just the success side.…