Menu

H100 vs A100 vs L40S: what to check before renting a cloud GPU
📰

H100 vs A100 vs L40S: what to check before renting a cloud GPU

DEV Community: cloud·AskGPU·20 days ago
#HnKPw8JN
#devtocloud#devto#dev#l40s#cloud
Reading 0:00
15s threshold

Renting a cloud GPU looks simple until the invoice arrives. Before you pick between an H100, an A100 and an L40S, run through this checklist.

1. VRAM first, FLOPS second

H100: 80 GB (HBM3). A100: 40 GB or 80 GB variants. L40S: 48 GB (GDDR6).

If the model plus KV cache does not fit, speed is irrelevant. Check which A100 variant a listing actually offers.

2. Interconnect

Training across several GPUs needs NVLink or fast InfiniBand. Many cheap listings use PCIe only.

The L40S has no NVLink, so it suits inference and single-GPU fine-tuning better than multi-GPU training.

3. Egress and storage

Ask what moving your checkpoints and datasets out costs. Some providers charge per GB, others nothing.

Persistent volumes are often billed even when the instance is stopped.

4. Spot vs on-demand

Spot can be far cheaper but can be reclaimed with little warning. Only use it with frequent checkpointing.

On-demand suits long jobs that cannot restart, and anything serving users.

5.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.