I wanted a model that answered in my own format β not a general chatbot, just something that reliably turned messy input into the labels I cared about. Retraining a full model was out of reach, but LoRA on a single consumer card turned out to be very doable. Here is the process I followed, including the parts that broke.
This is written for someone with one GPU (8β24 GB of VRAM), a Linux or Windows box, and Python installed. You don't need a cluster or a big budget. You do need to be comfortable with a terminal.
What LoRA actually changes
LoRA keeps the base model frozen and bolts small adapter matrices onto selected layers. During training, only those adapters update, which is why the memory footprint stays small: you're not storing gradients and optimizer states for billions of weights, only for a thin slice of them.
The result is a checkpoint measured in tens of megabytes instead of gigabytes.β¦