A 7B model is the honest ceiling for LoRA fine-tuning on a 16 GB consumer card, not because anything bigger throws an error, but because a 14B model silently spills into system memory and runs 50 times slower without ever reporting a failure.

Key takeaways

7B is the real ceiling on an RTX 5070 Ti at LoRA rank 16, sequence length 512, batch size 1: 11,897.9 MiB peak, well inside the card's 16,303 MiB.

14B completed without a raised exception in both tested conditions, but peak VRAM read 22,189.8 MiB (136% of the card) and 17,470.9 MiB (107%) - both over the physical limit, through Windows' silent system-memory fallback.

The cost of that fallback is severe: 21.7 seconds per training step at 14B against 0.4 seconds at 7B, roughly 51 times slower, with no error message anywhere.

Gradient checkpointing is close to free at 7B and below - same or less time, meaningfully less memory - but does not rescue 14B: it leaves the model still over the limit and makes the step 59% slower, not faster.

The registered…