To fine-tune an embedding model with LoRA, freeze the base encoder, attach a small LoRA adapter to its linear layers, and train only the adapter with a contrastive loss on (query, relevant passage) pairs from your own domain. With Sentence Transformers this is a few lines: build a LoraConfig with task_type=FEATURE_EXTRACTION , call model.add_adapter() , and train with MultipleNegativesRankingLoss . A few hundred to a few thousand clean pairs is usually enough. The adapter comes out at a few megabytes. Afterwards you must re-embed your whole corpus, because the fine-tuned model's vectors are not comparable with the old ones.
Why LoRA fits embedding models
Full fine-tuning updates every weight and leaves you with a second full copy of the model. LoRA freezes the weight matrix W and learns a low-rank correction B·A on top of it, scaled by alpha / r . Only A and B train.
For an encoder that matters in three ways:
Small files.…