Part of my work in the AMD AI League by lablab.ai and AMD. #AMD #lablab
Most RAG demos answer a question and paste a list of "sources" underneath. That list is usually padding: the top five chunks, whether they helped or not. For the AMD AI League's RAG challenge the grading was stricter. You return the value and exactly the files that value needs: no more, no less. On top of that the corpus is messy company data and the whole system has to run offline on an AMD GPU.
These are the design decisions that mattered.
The corpus is hostile
The input folder mixed PDFs (some scanned, some encrypted), DOCX, PPTX, XLSX, CSV, logs, source code and screenshots. Two rules kept the indexer alive:
Every file gets its own try . One corrupt PDF must never cost you the rest of the corpus. Unreadable, encrypted and unknown files are logged and skipped.
An encrypted file is never a source.…