This is Part 3 of my series on the Microsoft Agent Framework. You can read the original post over on lukaswalter.dev .

Introduction: The Problem with LLM Latency

LLMs generate responses token by token, producing output one character or word at a time.

For complex questions, such as comparing electric guitar models in terms of sound, feel and use across different music genres, the AI needs more time to generate its response.

When an application blocks and waits for the model to finish before displaying anything, users often see only a loading screen for several seconds.…