Menu

Five few-shot examples nearly doubled the cost of my short LLM prompt
📰

Five few-shot examples nearly doubled the cost of my short LLM prompt

DEV Community: beginners·Andrey Altrouter·8 days ago
#2qlrroIM
#devtobeginners#devto#dev#examples#prompt
Reading 0:00
15s threshold

You write a prompt that turns messy expense notes into JSON. The model gets the format slightly wrong once, so you add an example of a note and the JSON you expect. Then a second one for euros, a third for dates, and two more for luck. The output looks great and the prompt ships. What doesn't show up in testing is that those five examples get sent and billed again on every call, and in a short prompt they end up costing more than the task itself.

What you're paying for

Model APIs bill per token , a chunk of text about three-quarters of an English word. Input tokens are everything you send; output tokens are what the model writes back, and they're usually priced five or six times higher.

A few-shot example is a worked sample in the prompt: an input plus the answer you want, so the model can copy the pattern. A prompt with no examples, just instructions, is called zero-shot . The API keeps no memory between calls, so examples aren't "learned" once.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.