We're a platform team currently running RKE1 clusters with Canal (Flannel + Calico) as our CNI. Planning an RKE2 migration and evaluating whether to stick with Canal or move to Cilium. Looking for real-world experiences.
Our current setup:
RKE1 clusters managed via Rancher
Canal CNI (Flannel for VXLAN routing, Calico for network policy)
kube-proxy in iptables mode
Multiple clusters across different datacenters
What's pushing us to consider Cilium:
We recently had a node that was silently broken for 253 days. The Canal pod was healthy, passed all health checks, but the flannel masquerade rules in the iptables NAT chain had been wiped — likely by config management (Puppet). Every pod on that node could talk in-cluster but nothing could reach external services.…