I built a gateway that puts four Chinese model providers behind one OpenAI-compatible endpoint. Not because it is clever. Because wiring up four vendors by hand got old fast, and every vendor describes the same model differently.

Here is what actually broke.

Timeouts are not one thing. Our first version used one 90-second timeout for the whole request. That is wrong the moment you have a fallback. If provider A hangs for 80 seconds and then fails, your fallback gets 10 seconds and dies too. Now each attempt gets its own deadline, and the caller gets a clear 504 instead of a mystery 500.

Empty 200s are worse than errors. A provider can return HTTP 200 with an empty body. Our health check counted that as success for a week. Not sure how nobody noticed, tbh. Now we parse the body, and an empty completion counts as a failure.

Model lists drift between deploys. A model silently disappeared from one vendor's models endpoint. Our check passed because it only verified auth.…