Have you ever opened a drawer looking for one screwdriver and found a few hundred other things crammed in there? With ten things in the drawer you grab the right one without thinking. With a few hundred, you start digging, and sooner or later you grab the wrong one.
Agents are heading into the same drawer. MCP has made tools so easy to install that a few servers can hand your agent hundreds of them, and on every request it has to answer the same question first: which one do I use? The usual advice is to put all the tools in the prompt while they fit, and switch to retrieval once they don't. What nobody tells you is what happens after that switch. Does retrieval hold up as you keep adding tools, or does it slowly fall apart? And does paying for a bigger model help?
That's what I wanted to find out. So I took 200 questions from ToolRet [1], a public benchmark for tool retrieval, and kept them exactly the same while I kept making the pile bigger, from 1,000 tools all the way up to 3,794.…