Menu

FastAPI Latency Spikes Under Load: How One Blocking Call in an async Endpoint Stalls Every Request
📰

FastAPI Latency Spikes Under Load: How One Blocking Call in an async Endpoint Stalls Every Request

DEV Community: backend·Libme·2 days ago
#QrirzZ1L
#devtobackend#devto#dev#async#loop
Reading 0:00
15s threshold

TL;DR: An async def endpoint runs directly on the event loop. If it calls anything that blocks ( time.sleep , requests.get , a sync database driver, heavy CPU work), the loop cannot serve any other request until that call returns. The fix is to either declare the endpoint with plain def so the framework runs it in a thread pool, push the blocking call into a thread with asyncio.to_thread , or switch to a genuinely async client. The symptom that sends people searching is odd: health checks time out and p99 explodes while CPU looks fine.

What does the symptom look like?

The pattern I see over and over: an endpoint is fast in isolation, and a load test or a traffic burst makes unrelated endpoints slow, including /health . Uptime probes fail, the load balancer marks the instance unhealthy, and the instance restarts under load, which makes things worse.

Here is a minimal reproduction.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.