Menu

One PDF endpoint in our async service made every other endpoint on the worker wait
📰

One PDF endpoint in our async service made every other endpoint on the worker wait

DEV Community: fastapi·Sergey Shinder·20 minutes ago
#eW4kInjP
#devtofastapi#devto#dev#worker#endpoint
Reading 0:00
15s threshold

Our delivery slot service is a FastAPI application under uvicorn with four worker processes. Most of its endpoints are small: is this slot available, reserve it, release it, each answering in a few milliseconds. In August we added one more, which renders a delivery note as a PDF for the depot office, and wrote it as async def like everything else in the file.

From the next morning, slot lookups had a p99 of over two seconds between seven and eight, and normal latency for the rest of the day. The dashboard broke latency down by endpoint and pointed at availability, so we spent a week on its database queries and found nothing wrong with them.

An async def endpoint runs on the worker's event loop, one thread that interleaves every request in flight by switching between them whenever one awaits something. The PDF library is synchronous and pure Python, and a delivery note takes it between one and two seconds of CPU. It never awaits anything, so for those seconds the event loop runs nothing else.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.