AssemblyAI Cut Its Speech-to-Text GPU Bill by 70% — by Letting AWS Kill Its Machines
Every AI vendor will tell you inference is expensive. AssemblyAI actually did something about it: it runs its speech-to-text fleet on EC2 Spot Instances — the cheap compute AWS can yank with two minutes' notice — and still hits its SLAs. The trick isn't a better model. It's an architecture that treats being evicted as a normal Tuesday.
The 70% number is real — and narrower than it sounds
AssemblyAI reports 70% lower inference compute costs versus On-Demand, measured on its own production fleet while compute demand grew nearly fourfold year over year. That's a real, audited-scale result. But note the wording: inference compute , not total platform cost. Nobody quotes a whole-platform saving here, and anyone who repeats "70% cheaper" without that qualifier is overselling the case study.
The queue is the whole design
The core move is decoupling submission from execution.…