Now that we understand what RAG is and why it is so popular, let’s understand how RAG actually works in real systems.
So now we will talk about the complete RAG pipeline .
RAG pipeline has two main components:
First is called the ingestion pipeline
Second is called the retrieval pipeline
These are very important concepts, but they are actually very simple once you understand them.
1. Ingestion Pipeline
Let’s first understand the ingestion pipeline.
Ingestion pipeline basically means how we prepare the data before giving it to the model.
You can think of it like preparing an open book before the exam.
We first prepare all the information in a structured format so that the model can later access it easily.
Step 1: Collect Data
First we collect all the data.
This data can be anything like:
PDF files
simple documents
Excel files
websites
or even an entire company’s internal database
So basically, we take all possible sources of information.
Step 2: Extract Useful Data
Once we collect the…