Cross-posted from otherthings.cloud , where the rest of my homelab notes live.
The last post ended by saying NoOverlay mode was next. This is that post.
In OpenShift 4.22, a user-defined network can drop the overlay entirely. Pod traffic between nodes goes out as plain IP. Each node's pod subnet is advertised over BGP, and the network routes it like any other prefix.
The documentation gives one reason for it:
You can use no-overlay mode to route layer 3 pod traffic directly over the underlay network with BGP, which reduces encapsulation overhead and improves east-west performance.
Performance is the one thing I can't measure honestly. My cluster is three nested VMs on one Proxmox host. So this post leaves speed for later and asks the other questions: what goes on the wire, what the network sees, and what it took to get there.
TL;DR
With BGP on the node's primary NIC, a pod reaches the outside world with its own IP, and the outside world reaches it back, without any change to host settings.…