tl;dr: As of version 0.1.4 , the gearhash crate has gained a NEON backend which makes it roughly 2× faster on ARM64 at typical chunk sizes. It is selected automatically on aarch64 and backwards compatible, so consumers of the crate don't need to do more than just update. Read on if you're interested in the details of how this was achieved, or skip straight to the final results .

How it all started

At the end of 2019, I was building a personal backup system, and as part of this, became interested in a technique called content-defined chunking. The key idea behind it is that instead of chunking files on fixed chunk boundaries, you run a sliding window hash function across the file and trigger a chunk boundary whenever the hash has a particular value.…