Your semantic cache will happily replay the wrong command

Semantic caching works well for LLM responses because a near-match is usually good enough. Ask a slightly different question, get the cached answer, save a call. The failure mode is a mediocre answer.

Trajectory caching — caching the sequence of tool calls an agent makes, not just its final text — has a different failure mode. The cache doesn't store prose. It stores actions. And a near-match on actions can be the opposite operation.

Concrete example. An agent runs the goal "deactivate user 4471" and the trajectory gets cached. Later, a request comes in: "activate user 4471". With text-embedding-3-small, those two goals score 0.853 cosine similarity . A plain semantic cache running a 0.85 threshold — a common default — scores that as a hit and replays the deactivation trajectory in response to an activation request.

The embedding didn't fail. The two sentences are on the same topic, mention the same entity, and have the same structure.…