Dave Mille

Author Profile

Claim This Author Profile

Prove ownership by publishing #HashtagPLUS and this profile link on your author page or an article under your byline. A moderator or admin will review the request before it merges into your real HashtagPLUS username.

0 karma0 postsjoined 5 days ago

Notes from a 2 AM On-Call: Why Our vLLM Semantic Router Dropped 40% of Requests Under Load

🌐 dev.toSource

A production postmortem on silent request drops in vLLM semantic routing at scale, the async executor bottleneck we missed, and the 8-line fix.

#devtoopensource #devto #dev #route #router #requests #async #prompt

5 days ago

Menu

Dave Mille