Menu

1 / 3

How do you fine-tune Mistral 7B on your own data?

DEV Community: machinelearning·Viswaretas Kotra·11 days ago
#HNBi4MkN
Reading 0:00
15s threshold

To fine-tune Mistral 7B on your own data, train a LoRA adapter on the 16-bit model on one 48 GB GPU, then check it against held-out examples. On Nodus that is a single TrainingJob on the managed nodus/sft runtime with a hard dollar cap. The RTX A6000 it fits on is listed at $0.333/hr on the Nodus pricing page . You do not need an H100 for this model, and the arithmetic for when a faster GPU would save money is short.

How much GPU memory does Mistral 7B need for fine-tuning?

LoRA on the 16-bit base needs about 14.5 GB just to hold the frozen weights, so plan on a 48 GB card. We got that from the Mistral-7B-Instruct-v0.3 repository on Hugging Face , which lists 7,248,023,552 parameters stored in bf16.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.