Every tensor in PyTorch lives on a device: the CPU, an NVIDIA GPU ( cuda ), or an Apple Silicon GPU ( mps ). An operation runs on the device where its inputs are, and all its inputs must be on the same device. Running a model on a GPU therefore comes down to two things: putting the model's tensors on the GPU, and putting each batch of data on the same GPU before it is passed to the model. This article explains how .to() does both, what goes wrong when one of them is missed, and how to write code that runs unchanged on any machine.
Detecting the device
import torch
device = ( "cuda" if torch . cuda . is_available ( ) else "mps" if torch . backends . mps . is_available ( ) else "cpu" ) print ( "Using:" , device )
Enter fullscreen mode
Exit fullscreen mode
Output on a machine without a GPU:
Using: cpu
Enter fullscreen mode
Exit fullscreen mode
On Google Colab with a GPU runtime this prints cuda , and on a Mac with Apple Silicon it prints mps .β¦