25 million records. Distributed GPUs. Two vector databases. And still, the most difficult part of production RAG is not embedding generation.

A RAG prototype is easy to demo:

PDF → chunks → embeddings → vector DB → LLM → answer

Enter fullscreen mode

Exit fullscreen mode

But production is where the questions change.

What if a policy PDF is revised after it has been indexed? What if an ingestion worker retries the same batch? What if a user from Tenant A retrieves a chunk owned by Tenant B? What if the vector index returns a confident but obsolete paragraph?

Those failures do not disappear by adding GPUs.

I recently revisited AWS's article, Build a RAG data ingestion pipeline for large-scale ML workloads , published March 13, 2024. It is an interesting, substantial experiment.…