Menu

1 / 2

I Spent 3 Hours Watching My Benchmark Hang, Then 6 Seconds to Fix It

DEV Community·mote·6 months ago
#WQ0EBkWi
#phase#software#coding#index#memory
Reading 0:00
15s threshold

Three hours. That's how long bench_column_index ran before I realized it wasn't going anywhere.

I was preparing for moteDB v0.2.0 and running the usual performance suite. Twelve DB instances in parallel, each doing SELECT WHERE col = ? queries while a background thread built indexes. Queries that should take milliseconds started taking minutes. Then hours. Then nothing.

The culprit was a single RwLock<GenericBTree> protecting every read and write to the column index. When the background thread grabbed the write lock to bulk-insert, every query blocked. Simple as that. Twelve threads fighting over one lock.

Here's what I did about it — and how I got that 3-hour hang down to 6.6 seconds .

The Architecture That Was Killing Us

v0.1.7 had a straightforward design: one B-Tree, one RwLock. Clean. Wrong.

SELECT WHERE col = ?…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.