AI traffic does not look like ordinary web traffic. Requests carry multi-megabyte documents, and replies stream token by token over long-lived connections (Server-Sent Events). Network settings built for short page loads break that pattern. This lesson covers three places to tune: traffic coming in, traffic between services, and traffic going out.

In plain terms: Think of the cluster as a building. The front door must stay open long enough for a slow conversation, the corridors between rooms must not jam, and the exits must be locked except for the few doors you approve.

1. Ingress: keep long streams alive

An ingress controller (NGINX, Traefik or Envoy, for example) sits at the front door. Its defaults are tuned for quick requests and break AI streaming in two ways.

Timeouts. Default proxy read and send timeouts are often around 30 seconds, which can cut a connection while a model works through a large context. Raise them to several minutes for your slowest replies.

Buffering.…