We Cut API Latency 55% with Rust 1.88 and Tokio 1.40 for Our Core Service

Our core API service handles 12M daily requests across 40+ endpoints, powering everything from user auth to real-time analytics. For months, we’d been stuck with P99 latencies hovering around 180ms, well above our 100ms SLA. After evaluating multiple optimization paths, we migrated our core service from a Go-based stack to Rust 1.88 paired with Tokio 1.40 — and cut P99 latency by 55% to 81ms, with no loss in throughput.

Why We Chose Rust + Tokio

Our original Go service relied on goroutines and the standard net/http stack. While goroutines are lightweight, we hit two key bottlenecks: frequent GC pauses (up to 12ms per pause) and inefficient async I/O handling for our high-concurrent workload (8k+ concurrent connections per node).…