Menu

1 / 5

bitcask-kv: a log-structured key/value store in Rust, from the paper

DEV Community: rust·Abdeldjalil Chougui·about 9 hours ago
#pJPhaAgi
#devtorust#devto#dev#files#bitcask
Reading 0:00
15s threshold

A key/value store looks like the simplest database there is: put , get , delete . The hard part is not the API. It is what happens when the power goes out in the middle of a write, when the data files grow forever because old values are never reclaimed, and when the process restarts and must find every key again.

The Bitcask paper, written for the Riak database, answers all three with one idea: never modify data in place. Append every write to a log, keep an index in memory, and clean up in the background. The paper is a few pages long, which makes it a good first storage engine to build. bitcask-kv is my implementation in Rust: append-only data files, an in-memory hash index (the "keydir"), merge/compaction, hint files for fast startup, and crash recovery of torn writes. Here is what I built and what it teaches.

What I built

Append-only data files.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.