Menu

I ran the same QLoRA 7B job 8 times on Vast.ai RTX 4090s. The bill went from $0.19 to $0.81. Here is what moved it.
📰

I ran the same QLoRA 7B job 8 times on Vast.ai RTX 4090s. The bill went from $0.19 to $0.81. Here is what moved it.

DEV Community: cloud·Andy White·4 days ago
#yWZ4s3QV
#devtocloud#devto#dev#training#hosts
Reading 0:00
15s threshold

When people compare a rented RTX 4090 with a cloud A100, they multiply the hourly GPU price by the training time. I wanted to see

what actually shows up on the Vast.ai invoice for one fixed job on different hosts.

The job:

Qwen2.5-7B-Instruct, 4-bit QLoRA, yahma/alpaca-cleaned , 300 steps, checkpoint every 50 steps.

Image pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime , dependencies installed at start.

One RTX 4090, host reliability >= 0.98, network >= 500 Mbit/s.

Three things from the invoices:

Traffic: $0.81 against $0.19. On the first run the GPU hour looked cheap, but the bill was $0.81: $0.13 for the GPU and $0.66 for traffic, because the host charged for bandwidth and the job downloads a 15 GB model. After I started picking hosts by the full hourly price including expected traffic, the next run was billed $0.19 (0.44 h on the machine, about 19 minutes of it training).

Same GPU, different bill: $0.21 to $0.38.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.