Most fine-tunes need one GPU, full training of a 70B model needs 8 to 24, and pretraining needs as many as your deadline demands. On Nodus you ask for exactly that shape, from one RTX 4090 listed at $0.29/hr to several 8-GPU machines in one multi-node run (in Beta). The number to ask for is the larger of two floors: enough GPU memory to hold the training state, and enough GPU-hours per day to finish before your deadline.
We built this rule from published memory and compute formulas, and every table below is our own arithmetic, shown so you can redo it for your model. Our posts on VRAM for 70B models and how long training takes go deeper on each floor.
How do you work out how many GPUs a training run needs?
Compute two numbers and take the larger one. The memory floor is the fewest GPUs that can hold the model, gradients and optimizer state.…