Menu

Cite only what you need: building an offline document RAG on AMD ROCm
📰

Cite only what you need: building an offline document RAG on AMD ROCm

DEV Community: ai·Miguel Morente de las Heras·about 2 hours ago
#drLBNTM8
#devtoai#devto#dev#amd#lablab
Reading 0:00
15s threshold

Part of my work in the AMD AI League by lablab.ai and AMD. #AMD #lablab

Most RAG demos answer a question and paste a list of "sources" underneath. That list is usually padding: the top five chunks, whether they helped or not. For the AMD AI League's RAG challenge the grading was stricter. You return the value and exactly the files that value needs: no more, no less. On top of that the corpus is messy company data and the whole system has to run offline on an AMD GPU.

These are the design decisions that mattered.

The corpus is hostile

The input folder mixed PDFs (some scanned, some encrypted), DOCX, PPTX, XLSX, CSV, logs, source code and screenshots. Two rules kept the indexer alive:

Every file gets its own try . One corrupt PDF must never cost you the rest of the corpus. Unreadable, encrypted and unknown files are logged and skipped.

An encrypted file is never a source.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.