Community GPU marketplaces like Vast.ai rent an RTX 4090 for well under a dollar an hour. For a lot of fine-tuning
work that is the cheapest compute you can get. The catch is that the machine belongs to someone else: hosts reboot,
lose their network, get outbid, or simply disappear in the middle of your run.
I wanted to know two things: how much cheaper a real fine-tune is on such a machine, and what it takes for that
fine-tune to survive the host disappearing.
The job
QLoRA fine-tune of Qwen2.5-7B-Instruct (4-bit), yahma/alpaca-cleaned, 300 steps, checkpoint every 50 steps.
Public image pytorch/pytorch:2.5.1-cuda12.4-cudnn9-runtime , dependencies installed at start.
One RTX 4090, host reliability β₯ 0.98, network β₯ 500 Mbit/s.
What it cost
The number below is taken from the Vast.ai invoice for one run without failures, not from "price Γ time".
Time on the machine 0.44 h (about 19 min of training, the rest is start-up and the model download)
Billed by Vast.ai β $0.19
Same 0.44 h on one A100 atβ¦