Ask a chatbot a question about your own documentation and it answers confidently, fluently, and sometimes completely wrong. The model is not broken. It is doing exactly what it was trained to do: produce plausible text. Retrieval-augmented generation, or RAG, exists to change the job from "produce plausible text" to "answer from these documents." When RAG works, hallucinations drop sharply. When it fails, it usually fails in one of a handful of predictable places, and each one is fixable.

What RAG actually is

A RAG pipeline has four stages, and the model is only the last one. First, your documents are split into chunks. Second, each chunk is converted into a vector embedding, a list of numbers that captures its meaning. Third, at query time, the user's question is embedded the same way and the nearest chunks are retrieved.…