I have been thinking about how result tables get read when the interesting part is how close everyone is.

The Agent Memory Leaderboard's public board from August is a nice case. On the Cycle 1 coding board, Open-source Methods, 43 listed systems run from 46.00% to 52.67% solve rate over 150 tasks, meaning 69 to 79 tasks solved, and eight systems sit together at 52.67%. Since one task there is worth 0.67 percentage points, the whole top cluster moves by a task at a time.

Honestly I like seeing this. My read is that a lot of different designs land in the same neighbourhood on these coding tasks, and convergence like that is informative in itself. It also means the table format carries real weight.…