The order of middleware affects what an AI application measures and repeats. Placing telemetry outside a cache also observes calls served from the cache. Placing it inside a retry component records every attempt, including failures that the caller does not observe.

A dashboard might show one successful call even though the provider processed two requests. The worked examples below demonstrate how this occurs, using a support-reply operation and its recorded call sequences.

This article focuses on how middleware order changes observable behavior, rather than implementing a complete middleware library. The examples use selected code excerpts from a controlled experiment to explain the resulting event traces.

Read the pipeline from outside in

An IChatClient pipeline consists of nested clients. Each wrapper can call its inner client, return early, or throw an exception. A retry wrapper might call its inner client multiple times.

ChatClientBuilder places the first registered wrapper at the outermost layer.…