Menu

Post image 1
Post image 2
1 / 2
0

Chat vs. Streaming: Don't Keep Your Users Waiting

DEV Community·Lukas·5 months ago
#6ZIVv2yS
#dotnet#csharp#ai#agent#response#runasync
Reading 0:00
15s threshold

This is Part 3 of my series on the Microsoft Agent Framework. You can read the original post over on lukaswalter.dev . Introduction: The Problem with LLM Latency LLMs generate responses token by token, producing output one character or word at a time. For complex questions, such as comparing electric guitar models in terms of sound, feel and use across different music genres, the AI needs more time to generate its response. When an application blocks and waits for the model to finish before displaying anything, users often see only a loading screen for several seconds. This gap leads to a less satisfying user experience because the system lacks visual feedback that it is processing. The Standard Way: RunAsync (Blocking) The standard Microsoft Agent approach uses await agent.RunAsync("Your question") . With this method, the program execution pauses and waits until the AI has fully generated its response before continuing.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More