I need to transfer 350GB of data from my home to my parents' home.

@flux

> Not time to get them to upgrade?

There's no viable option for that currently, where they live.

@flux I've started the rsync

@ross

I want to say "Snowball", perhaps?

@flux @david_chisnall

My parents' VDSL download is the bottleneck here, but that's fine - it will be done when it is done, even if I have to pause it during the day to avoid saturating their line while they need to use it.

@RootMoose @flux

I *think* that that is most useful for individually large files, rather than a load of small files, as I have here. But useful all the same!

@bojanlandekic @flux @david_chisnall I am not aware of a way to do it adaptively by time (e.g. use full bandwidth during "night" hours, and throttle to 2Mbit during "day" hours), or any other factor come to think of it...

Ideas welcome!

@SforStuart That must be slower than my rsyncing it!

@SforStuart My parents' VDSL line is running at about 12Mbit, so I might as well just push the files to them!

@SforStuart I'm definitely not the bottleneck here I'd be very surprised if they could download it from OneDrive faster than they can download it from me.

@neil @bojanlandekic @flux @david_chisnall Restrict it to 90% all the time.

@mansr @bojanlandekic @flux @david_chisnall

That would leave my parents with ~1Mbit, so not ideal!

@neil @bojanlandekic @flux @david_chisnall I suppose you could have a cron job restart the rsync with different options at different times. There are also ways of rate limiting at the kernel level.

@mansr @neil @bojanlandekic @david_chisnall Yes! Lots of good options. Wonder if rclone has some additional features that make it easier than rsync to manage.

@flux @neil @bojanlandekic @david_chisnall Additional features is something I've rarely wished for in rsync.

@neil @mansr @bojanlandekic @flux

TCP rate limits based on the delay of the acknowledgments and most consumer routers are pretty good at ensuring some modest fairness between streams.

The main failure mode is in the opposite direction with asymmetric home connections. If you’re doing a lot of uploads, the TCP window replies get delayed and that tanks the download speed.

@neil This reminds me of a visit to Jodrell Bank in the early 2000s. They were syncing data between an international array of radio telescopes, and boasted that their fastest data transfer medium was FedEx.

@kbm0 Yep!

@neil Here’s me, preparing to transfer half a terabyte from our data center to the office. 20 mins or so, less if I ignored a few speed limits.

@neil A few years ago, a friend of mine used to use the phrase "sneakernet" for that.

Sometimes sneakernet is still the fastest way to move data.

I remember working for an org decades ago with little tiny WAN pipes when I first got there, people complained all the time, it never met expectations. I recall doing the math for them to give them some kind of mental picture.

Our most remote building was about 5 mi / 8 km from town. If you had to move more than about a CD's worth of data it was faster to walk it over.