In 2026, most teams managing multiple LLM integrations are running into the same wall: their AI routing layer was designed for one provider, one protocol, and one use case. Adding a second provider means patching. Adding voice or MCP (Model Context Protocol) support means forking. The gateway that worked fine at the prototype stage becomes the thing that slows every subsequent decision. Respan, an open-source AI gateway written in Rust, is one of the more architecturally honest answers to this problem I've seen this year.
This is not a product review. It's a retrospective on what we set out to build when we first tried to wire multiple LLM providers into a single routing layer, what broke, and why the architectural choices Respan makes are worth studying regardless of whether you adopt it directly.
What We Set Out to Build
The original goal was straightforward: a single gateway that could route requests to different reasoning models depending on task type.…