Google’s tensor processing units (TPUs) — its custom AI chips — now serve AI in production at Meta (creator of the widely used PyTorch AI framework) through a new PyTorch-native version of the SGLang inference engine, announced on Google’s developer blog. A matching version of vLLM, another widely used inference engine, was also announced as open source.

Previously, both engines used Google’s own JAX AI framework to run on TPUs. The new versions allow the hardware to be a native option for PyTorch, mostly without rewriting.