Renting a cloud GPU looks simple until the invoice arrives. Before you pick between an H100, an A100 and an L40S, run through this checklist.
1. VRAM first, FLOPS second
H100: 80 GB (HBM3). A100: 40 GB or 80 GB variants. L40S: 48 GB (GDDR6).
If the model plus KV cache does not fit, speed is irrelevant. Check which A100 variant a listing actually offers.
2. Interconnect
Training across several GPUs needs NVLink or fast InfiniBand. Many cheap listings use PCIe only.
The L40S has no NVLink, so it suits inference and single-GPU fine-tuning better than multi-GPU training.
3. Egress and storage
Ask what moving your checkpoints and datasets out costs. Some providers charge per GB, others nothing.
Persistent volumes are often billed even when the instance is stopped.
4. Spot vs on-demand
Spot can be far cheaper but can be reclaimed with little warning. Only use it with frequent checkpointing.
On-demand suits long jobs that cannot restart, and anything serving users.
5.…