Google Cloud just changed the AI hardware game. At Google Cloud Next 2026, the company unveiled its eighth-generation Tensor Processing Units, splitting them into two distinct chips for the first time: TPU 8t for massive model training, TPU 8i for lightning-fast inference. This move acknowledges a hard truth. Training and serving demands have diverged sharply.

Amin Vahdat, Google Cloud’s SVP and chief technologist for AI infrastructure, put it bluntly: “the demands of training and serving have completely diverged.” Google Cloud Blog . And so, Google built separate powerhouses.

TPUs trace back over a decade. Google designed these custom ASICs to crush matrix math—the core of neural networks. Unlike general-purpose GPUs, TPUs use systolic arrays for efficient, high-throughput computations on AI workloads. Early versions like TPU v1 handled inference inside Google services. Pods scaled to thousands of chips. By v5e, each chip hit 393 trillion int8 operations per second.…