You can now sort the providers behind a model by cost, time to first token (TTFT), or throughput (TPS) in AI Gateway .

The default provider order blends provider reliability, quality of model output, cost, and speed of response. You can now use sort for explicit control over ranking criteria.

For models with many providers and noticeable cost or speed variation, you can use sort to optimize on your dimension of choice. Ranking is computed at request time, so newly added providers, price changes, and shifts in observed latency or throughput flow through automatically without any code changes.

Set sort on providerOptions.gateway to one of the three values:

Value

Description

Direction

When to use

'cost'

Sort by the provider's listed input price per million tokens

Lowest price first

High-volume, cost-sensitive work

'ttft'

Sort by median time to first token, in ms

Lowest latency first

Latency-sensitive workloads where response speed matters

'tps'

Sort by median tokens per second…