On 19 October, AWS released TorchNeuron, which lets PyTorch run directly on its Trainium chips. A day later, Google and Meta released TorchTPU, which does the same for Google's TPUs. Both arrived at the PyTorch Conference, both are open source, and both connect to PyTorch through the same small hook.

That last part is the story. Two companies with no reason to coordinate reached the same conclusion: the road onto a working engineer's laptop runs through one extension point in PyTorch.

What happens when you call .to("cuda")

Write a + b on two tensors and PyTorch does not immediately add anything. It looks up a name, aten::add , in a table.

The part doing that lookup is called the dispatcher. It checks where the tensors live, reads a label off them — CPU, CUDA, and so on — and uses it to pick the function that actually does the addition on that hardware. Those hardware-specific functions are called kernels.…