The problem: the five-line feature that metastasizes

Every team adds their first LLM feature the same way. Someone writes five lines inside the existing codebase:

client = OpenAI ( api_key = os . getenv ( " OPENAI_API_KEY " )) summary = client . chat . completions . create (...)

Enter fullscreen mode

Exit fullscreen mode

It works. Then product asks for a second feature — chat. Then classification for the support inbox. Six months later, OpenAI calls are scattered across four modules, the API key lives in three config files, nobody knows which endpoint is burning tokens, and swapping models means a search-and-replace across the whole repo. The LLM stopped being a feature and became infrastructure — except nobody built it like infrastructure.

There's a boring pattern that prevents this: put LLM calls behind a tiny HTTP service of your own. Your app talks to one small FastAPI sidecar; the sidecar talks to the provider.…