Cross-posted from the ByteChef blog . Part 7 of the AI Agentic Patterns in ByteChef series.

Every chat agent has the same slow leak. Each message grows the history, and the whole history goes into the next prompt. Long chats get slower, cost more and eventually hit the model's context limit.

The usual fix is "keep the last N messages". It works fine until the agent uses tools.

The problem with "last N messages"

A tool call and its result are two separate messages. Trim the history between them and the model gets one without the other: a result with no call, or a call with no result.

user: What's the status of order 4512? assistant: [tool call] getOrder(4512) <- window cuts here tool: { "status": "shipped" } <- this survives assistant: It shipped yesterday.

Enter fullscreen mode

Exit fullscreen mode

Nothing throws.…