Menu

1 / 2

How I track per-customer LLM costs in production

DEV Community·John Medina·6 months ago
#j9enqDUL
#llm#opensource#ai#costtracking#openai
Reading 0:00
15s threshold

Tracking LLM costs across an entire app is easy. Finding out which customer is actually burning through your OpenAI bill? That's a nightmare.

For a while, we were just eating the cost. You look at the Stripe dashboard, look at the OpenAI invoice, and pray the margins make sense. But when a single power user decides to process 10k documents through Claude on a Saturday night, averages stop mattering real fast.

tbh, most billing dashboards are useless for this. They tell you you spent $400 yesterday, but not who spent it.

Here is what actually works in production, without over-engineering your entire stack.

The Metadata trick

If you're using OpenAI or Anthropic, you can pass metadata with every request. Don't just log it in your db. Pass the user_id or tenant_id directly to the provider.

OpenAI supports this natively. Anthropic has custom headers. OpenRouter makes it trivial.

const response = await openai . chat . completions .…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.