An API model costs nothing until the bill arrives, and a local model costs everything up front: hardware, quantisation loss, engineering time, before it answers a single question. Which one is cheaper depends on volume and constraints, not on which sounds more modern.

Key takeaways

The break-even point between API and local hosting depends on request volume and GPU utilisation, not a fixed rule of thumb. A card that sits idle most of the day never pays for itself, however cheap its per-token math looks on a spreadsheet.

Real numbers from a registered quantisation experiment on an RTX 5070 Ti (16GB): Q4_K_M quantisation delivered 2.22x the throughput of fp16 with 0.9835 average embedding similarity, a genuine trade, not a free lunch.

API providers get batching economies of scale that a single-GPU local deployment cannot replicate at low request volume, so a naive per-token price comparison is not comparing like with like.

Privacy and data residency requirements can override the entire cost calculation.…