Menu

Regional inference now available on AI Gateway - Vercel
📰

Regional inference now available on AI Gateway - Vercel

Vercel News·Walter Korman·5 days ago
#9sb2RP7j
#vercel#request#provider#regional#gateway
Reading 0:00
15s threshold

AI Gateway now supports regional inference . Set inferenceRegion on a request to pin it to the US or EU. Every model provider that supports the selected region handles it the same way. Inference runs there, and any data the provider keeps is stored there.

AI Gateway supports two pinned regions, plus global routing:

If no model provider can serve it, the request fails rather than running somewhere else. Every response reports the region that served it, so you can confirm where each request ran.

Here's a request pinned to the US with the AI SDK:

import { streamText } from 'ai' ;

const result = streamText ( {

model : 'moonshotai/kimi-k3' ,

prompt : 'Summarize this internal document.' ,

providerOptions : {

gateway : {

inferenceRegion : { scope : 'zone' , geoRegion : 'us' } ,

} ,

} ,

} ) ;

Until now, teams with data residency or compliance requirements had to configure regional routing separately for every provider, with no reliable way to confirm where a request actually ran.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.