Menu

I Sorted Two Million Rows to Keep Ten
📰

I Sorted Two Million Rows to Keep Ten

DEV Community: dotnet·Sukhpinder Singh·1 day ago
#TnKDo3AG
#devtodotnet#devto#dev#heap#rows
Reading 0:00
15s threshold

A leaderboard endpoint I wrote returns the ten highest-scoring players, and for a while I never thought about it twice. Ten rows out, done. Then it started showing up in our gen0 collection graphs during traffic spikes, which made no sense to me, because the response is ten rows. The allocation counter disagreed, and that counter has never once flattered me.

Here's the line that did it, more or less:

var top = players . OrderByDescending ( p => p . Score ) . Take ( 10 ) . ToList ( ) ;

Enter fullscreen mode

Exit fullscreen mode

It reads exactly like what I want. Sort by score, take ten. The problem is the order of operations under the hood. OrderByDescending doesn't know I only want ten until Take shows up later in the chain, so it buffers every player into an internal array first, then sorts enough of it to hand ten back. To return ten rows, it holds all of them at once. On a small table nobody notices.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.