Menu

LoRA fine-tuning on one GPU: from JSONL to a working adapter
πŸ“°

LoRA fine-tuning on one GPU: from JSONL to a working adapter

DEV Community: machinelearningΒ·AI ReleaseΒ·about 13 hours ago
#cAOaK9Hw
#devtomachinelearning#education#devto#dev#model
Reading 0:00
15s threshold

I wanted a model that answered in my own format β€” not a general chatbot, just something that reliably turned messy input into the labels I cared about. Retraining a full model was out of reach, but LoRA on a single consumer card turned out to be very doable. Here is the process I followed, including the parts that broke.

This is written for someone with one GPU (8–24 GB of VRAM), a Linux or Windows box, and Python installed. You don't need a cluster or a big budget. You do need to be comfortable with a terminal.

What LoRA actually changes

LoRA keeps the base model frozen and bolts small adapter matrices onto selected layers. During training, only those adapters update, which is why the memory footprint stays small: you're not storing gradients and optimizer states for billions of weights, only for a thin slice of them.

The result is a checkpoint measured in tens of megabytes instead of gigabytes.…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.