A token budget that only gets checked after the response comes back gets checked too late. By then the model has already run and the money is spent. If many requests from one tenant arrive in the same second, each of them reads the same "remaining budget" value, each of them passes, and the overspend shows up in tomorrow's cost report.

Release v2.5.0 of ai-runtime-platform adds an Inference Budget Guard to the gateway's chat completions path. It treats spend as admission control. Before a request reaches the model, the gateway reserves an estimated cost against a per-tenant or per-API-key window. When the request finishes, the reservation is settled against what actually happened.

Where it sits in the request path

The guard runs inside the gateway process.…