H100 & RTX 5090 GPU Cloud Effective Cost: AWS p5 vs RunPod vs Vast.ai Cold-Start & Billing Granularity Audit

Deploying large language models (LLMs) or generative diffusion pipelines in production usually starts with a spreadsheet: you multiply the hourly rate of an NVIDIA H100 SXM5 or RTX 4090/5090 by 730 hours and calculate your monthly run-rate.

Then your first cloud invoice arrives, and the figure is 2.5x higher than your model predicted.

The discrepancy stems from three architectural cost drivers that hyperscalers omit from their headline rate:

Cold-Start Billing Waste : When auto-scaling from zero, you pay for 15-minute container pull and CUDA weight initialization cycles before serving a single token.

Billing Granularity Inefficiencies : Minimum billing increments (e.g., AWS EC2 rounding up to full hours or 10-minute minimums) heavily penalize bursty inference workloads.

Egress & Storage Leaks : High-speed NVMe attach fees ($0.30/GB/month) and high egress rates ($0.09/GB on AWS) quietly erode…