Menu

6 Things That Quietly Break Your LLM Prompt Cache in .NET
📰

6 Things That Quietly Break Your LLM Prompt Cache in .NET

DEV Community: dotnet·MICHAEL-MAURICE·1 day ago
#GZDKOsmr
#devtodotnet#devto#dev#prefix#cache
Reading 0:00
15s threshold

I had a support bot that sent the same 1,300-token system prompt on every request, and I assumed the provider's prompt cache was doing its job. OpenAI and Azure OpenAI cache long prompt prefixes automatically, and for the models I priced, cached input costs a tenth of normal input. Free money, as long as the cache actually hits.

Mine didn't. When I finally measured it, only 7% of my input tokens were cached. The cause was one friendly line at the top of the system prompt: Today is 2026-11-13 09:14:07 UTC. You are helping Acme Robotics.

That line fixed nothing and broke two things at once. It's also not the only way to lose the cache without noticing. Below are the six cache breakers I now check for in every .NET LLM app, how each one shows up, and the shape of the fix.

Tech Skill Builder is $5/month for a very limited time only (normally $10). Grab it now before it goes back to $10.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.