Menu

Asking an LLM "are these two tickets duplicates?" costs $111 for 1,000 tickets and $11,130 for 10,000
📰

Asking an LLM "are these two tickets duplicates?" costs $111 for 1,000 tickets and $11,130 for 10,000

DEV Community: api·Andrey Altrouter·4 days ago
#ou8BNsvo
#devtoapi#devto#dev#tickets#pairs
Reading 0:00
15s threshold

You have a pile of support tickets and want to find the ones that describe the same problem. Models are good at this: show one two tickets, ask "same issue, yes or no?", and the answer is usually right. So you write a loop that compares each ticket with every other one. It costs a dollar on your 100 test tickets, and you ship it. Then it runs on the real backlog.

Why the bill grows faster than the data

Model APIs bill per token , a chunk of text that averages about three-quarters of an English word. Input tokens are what you send, and output tokens are what the model writes back. A yes/no comparison is almost all input: two tickets plus a short instruction in, one word out.

The catch is the number of calls. With N tickets, the number of distinct pairs (each ticket matched with each other ticket once) is N × (N − 1) / 2. That's 4,950 pairs for 100 tickets, 499,500 for 1,000, and almost 50 million for 10,000. Ten times the tickets means about a hundred times the calls.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.