Menu

Sticky Sessions Won't Save Your AI Chat: 6 Questions Before You Run a .NET Chat Backend on Two Nodes
📰

Sticky Sessions Won't Save Your AI Chat: 6 Questions Before You Run a .NET Chat Backend on Two Nodes

DEV Community: dotnet·MICHAEL-MAURICE·about 10 hours ago
#fppAvBiS
#devtodotnet#devto#dev#answer#node
Reading 0:00
15s threshold

Scaling a normal API is a solved problem: make the servers stateless, put the state in a database, add nodes. An AI chat backend follows the same recipe and still goes wrong, because a chat turn is slow, costs money, depends on history, and gets retried by clients all the time.

When I moved a C# chat API from one node to three, nothing crashed. Every response was a 200. And the bill and the answers were both quietly wrong.

So instead of a tutorial, here are the six questions I now ask before a chat backend gets a second node, with the shape of the answer to each.

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10. The complete, tested project behind this post (a .NET 10 multi-node chat backend with Clean Architecture and 198 passing tests, running offline against a simulated model) plus a step-by-step PDF guide is inside.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.