Menu

Streaming LLM output through a Next.js proxy: why you only get stream_end
πŸ“°

Streaming LLM output through a Next.js proxy: why you only get stream_end

DEV Community: nextjsΒ·Ali AhmedΒ·11 days ago
#7Vb2q8gL
#devtonextjs#devto#dev#client#proxy
Reading 0:00
15s threshold

When you proxy an LLM backend's Server-Sent Events (SSE) through a Next.js route handler, the usual failure is that nothing arrives until the end, and then everything arrives at once, or the client only ever sees the final stream_end . Something between the backend and the browser is buffering . The fix is a handful of specific things: the Node.js runtime, dynamic = 'force-dynamic' , the right headers ( Cache-Control: no-cache, no-transform and X-Accel-Buffering: no ), passing bytes through as they arrive, and a client that parses SSE frames properly. I learned these building Turbo Alan Refiner , a multi-pass LLM text-refinement app ( frontend source ).

Why proxy SSE through Next.js at all?

Routing every backend call through /api/* route handlers avoids CORS, keeps tokens server-side, and lets you normalise the request.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.