For two days I kept telling anyone who would listen that my endpoint was slow. Then I opened my own timing code and realized I had been measuring three unrelated things and calling the sum "latency." Is the model slow, is the queue slow, or is my client just bad at arithmetic? Those are different problems with different fixes, and I had been treating them as one.
These are field notes from a 48-hour window, not a benchmark report. I am deliberately not quoting latency numbers here, because a number measured on my connection, my prompt, and my region tells you almost nothing about yours. What I can hand over is the harness, the mistakes, and the decision table I now use before blaming anyone's infrastructure.
The question I actually had
My vague complaint was "responses feel slow," which is not a measurable statement and therefore not a debuggable one. What I actually needed to know was where the wall-clock time went between pressing enter and seeing the last character.β¦