A chatbot request takes a few seconds. A phone call takes minutes, and during those minutes it holds a telephony connection, a speech-to-text stream, LLM calls and text-to-speech output, all of which have their own limits and failure modes.
Here are the lessons that matter most once an AI voice system goes from a handful of test calls to thousands a day.
1. A call is a session, not a request
Web backends are built around short requests. Voice calls are long-lived sessions with state: who's on the line, what's been said, which step of the conversation you're in. Keep that state outside the process handling the audio (Redis works well), so a worker restart or a reconnect doesn't lose the conversation.
2. Limit concurrent calls explicitly
Your telephony provider, your speech services and your LLM provider all cap how much you can run at once.…