Menu

I Route Every AI Agent to the Cheapest Model with Quota. Here's What Broke
πŸ“°

I Route Every AI Agent to the Cheapest Model with Quota. Here's What Broke

DEV Community: pythonΒ·GalaΒ·about 2 hours ago
#xiTlhTV0
#devtopython#devto#dev#model#router
Reading 0:00
15s threshold

I wanted coding with AI agents to be genuinely cheap, so I built a tiny Python router that sends every task to the cheapest model that still has quota left.

Within three days, a cheap model wrote a test that looped forever and ate 9 GB of RAM, another stopped halfway with "Free usage limit reached", and a third dumped 419 lines of duplicated helper functions into a script because it assumed "the grader supplies tests." πŸ˜…

Everyone is asking whether coding with AI is actually faster and cheaper. My honest answer? It's fast and nearly free, right up until a cheap model writes something that takes two hours to unwind.

So here's the router, the rules that came out of those crashes, and why it's forbidden from upgrading itself.

TL;DR:

Models live in three tiers (small, medium, large), each sorted cheapest first.

The router skips paid models and anything above 90% of its quota window.

If a model dies in its first 60 seconds, it tries the next one in the same tier.

It never escalates to a bigger tier on its…

Continue reading β€” create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.