📰00Read later0Read MoreVector Search Latency: How to Debug Go Spikes Under LoadDEV Community: go·ZebedeeHolloway9023·7 days ago#eQOYIHU1#devtogo#devto#dev#shape#top_k+5 more🧰Tag tools✨Add tagThe first move when vector search latency spikes under load is to reduce top_k and stop returning...15s
📰00Read later0Read MoreGoing from top-5 to top-20 retrieval makes every RAG answer 2.7x more expensiveDEV Community: beginners·Andrey Altrouter·10 days ago#WQN0wiUs#devtobeginners#devto#dev#answer#chunks+5 more🧰Tag tools✨Add tagIn a RAG bot, the question is a fraction of a percent of the bill. Most of it is the chunks you paste in to answer it. At 500 tokens a chunk, top_k = 20 instead of 5 makes every answer 2.7 to 2.8 times more expensive on any of five popular models.… Read more15s