Menu

Meter Cost per Accepted Answer, Not per Call
📰

Meter Cost per Accepted Answer, Not per Call

DEV Community: node·kongkong·9 days ago
#MgrxFhLR
#devtonodejs#devto#dev#meter#const
Reading 0:00
15s threshold

Meter Cost per Accepted Answer, Not per Call

Last Tuesday a merged pull request fired the "Draft changelog" button in our internal release dashboard, and the model answered in about four seconds with JSON that read fine to a human. It failed one layer later, inside the worker's schema check, because body came back as an array of strings instead of a string. My retry loop then did exactly what I had told it to do: it appended the validation error to the same conversation and asked again, which is a reliable way to make attempt two cost more than attempt one. My dashboard said everything was healthy, because it measured tokens per call.

I want to argue a specific, unhedged position here: the billable unit of an AI feature is the accepted result, and per-call metrics are a form of self-deception that happens to be the easiest thing to build. Every rejected attempt is a small number that dissolves into a large average, so the chart stays flat while the thing you actually pay for quietly gets worse.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.