A 1.18 TB copy between two seedboxes in the same data centre took 14.5 hours . The network between them did 8 Gbit/s on a single iperf3 stream. The ping was 0.29 ms. My tool averaged 21.6 MiB/s.
The tool is Hopsync , an open-source Rust service I'm building for moving files between SSH servers from a web UI or a CLI. The other side only needs a normal SSH account with SFTP. Its main trick is parallel "lanes": several SSH connections, with big files split across them. Over a 200 ms link that roughly doubles scp's speed, and on 300 small files it finishes in 39 s where scp takes 249 s.
On two hard drives sitting next to each other, the same trick made it slower than rclone. Here's why, and what fixed it.
Measure the ceiling first
Before touching code I wanted to know what "fast" meant on these boxes.…