An AI chat endpoint feels finished the moment tokens start streaming into the browser. Then your first real bill arrives, and the token count is higher than what your users actually read. The usual reason: responses that keep generating after the user has already closed the tab.
Where the money leaks
In the Next.js App Router, a streaming route opens a connection to the model and pipes tokens back to the browser. If the user navigates away or closes the tab mid-response, the browser drops its connection β but the upstream request to the model keeps generating to completion. You pay for every token it produces, and nobody ever sees them. On long answers with an impatient audience, that waste adds up fast.
The fix: stop generating when the browser disconnects
When you build the SSE stream yourself with a ReadableStream, its cancel() callback fires the moment the client connection drops.β¦