The meeting had moved. The search results had not caught up.

In an early memory-retrieval test for Sipplay, the query asked when the product review had finally been scheduled and which room it would use. Two cards looked relevant: the original Thursday morning booking, and the updated Friday afternoon arrangement.

Two of the three embedding models put the old booking first. Both found the new one, too. It was sitting in second place.

That is a useful failure to keep around. A dashboard watching only whether the correct memory appeared in the top five would have called the query a success.

I work on Sipplay , an AI character chat and interactive storytelling product. This article draws on our internal memory evaluation report. These are development-set results, not an independent benchmark or a claim that the final production system has passed validation.

Why we tested retrieval separately

When an AI character appears to forget something, several things may have gone wrong.…