Menu

I Timed Three Ways to Deduplicate an Array, Only One Scales
📰

I Timed Three Ways to Deduplicate an Array, Only One Scales

DEV Community: productivity·RAXXO Studios·2 days ago
#kpmxf9ce
#devtoproductivity#devto#dev#array#input
Reading 0:00
15s threshold

I benchmarked three ways to dedupe an array in Node 22, at sizes from 500 to 40,000 items

A Set-based dedupe took 4.27ms at 40,000 items, filter plus indexOf took 1,229.68ms on the same input

The gap is invisible under 2,000 items, then widens fast once one approach turns out to be O(n squared)

One of the three is the only version I would ship today, the other two I'm done writing from memory

I keep a small pile of "obviously fine" utility functions that nobody questions until a dataset grows past the size it was written for. Array deduplication is one of them, so I actually timed the three ways I have written it over the years instead of trusting my memory of which one was fast.

The Three Ways I Actually Wrote This

The first is the one most people reach for without thinking: [...new Set(arr)] . It builds a Set, which only keeps unique values, then spreads it back into an array.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.