Menu

I built an open-source file sync engine using Kotlin Coroutines & Java NIO that beats rsync by 40x on massive directories.

Reddit r/Kotlin·u/GuanTouYu·6 months ago
#1xGC6Uif
#rsync#janice#file#sync#massive
Reading 0:00
15s threshold

Hi everyone,

I’ve been developing Janice , a high-performance asynchronous file synchronization tool built from the ground up to solve a very specific bottleneck: the massive overhead when syncing directories with hundreds of thousands of files.

The Problem & The Goal: Traditional tools like rsync are fantastic, but they often choke on file traversal during incremental syncs. My primary engineering goal for this project was to ensure it can process TB-level data at full disk rate without the CPU or thread management becoming the bottleneck.

The Architecture: To achieve this, I abandoned traditional threading models. Janice leverages Kotlin Coroutines alongside Java NIO's asynchronous file and networking capabilities.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.