Introduction
Building an AI application that can retrieve company information and generate useful responses is exciting. But getting the model to produce a relevant answer is not the only engineering challenge.
There is another question that matters just as much:
How do we control what users send to the model and what the model is allowed to return?
A Retrieval-Augmented Generation (RAG) application can retrieve relevant company documents, pass them to a foundation model, and generate a response. However, retrieval and generation alone do not guarantee that every interaction is appropriate, safe, or aligned with company policies.
A user could submit a harmful prompt, attempt to override the application's instructions, or include sensitive personal information.…