Short answer: use staged retrieval with explicit collections, hard query budgets, and source context that survives every stage; tune recall inside those limits instead of letting a customer-support knowledge base search an unbounded PDF corpus.

For a B2B SaaS team answering questions over folders of customer manuals, the hard trade-off is retrieval quality versus tail latency. More candidates can improve recall, but they also enlarge reranking and answer context. My operating rule is to admit a bounded amount of work, preserve tenant and access-control metadata on every indexed item, and reject or degrade excess work before it becomes an invisible queue.

I've been paged by missed jobs and duplicate deliveries. The retrieval version of that incident has the same shape: retries arrive while the original work is still running, a shared worker pool fills, and a small latency problem becomes a backlog.…