←back to thread

Magic Wormhole: get things from one computer to another, safely

(github.com)
816 points tosh | 8 comments | 17 Aug 24 16:59 UTC | HN request time: 0.888s | source | bottom
Show context
geerlingguy ◴[17 Aug 24 18:23 UTC] No.41276702[source]▶
I've used this for years when passing large files between systems in weird network environments, it's almost always flawless.

For some more exotic testing, I was able to run my own magic wormhole relay[1], which let me tweak some things for faster/more reliable huge file copies. I still hate how often Google Drive will fall over when you throw a 10s-of-GB file at it.

[1] https://www.jeffgeerling.com/blog/2023/my-own-magic-wormhole...

replies(4): >>41277198 #>>41277682 #>>41277698 #>>41278657 #
bscphil ◴[17 Aug 24 20:28 UTC] No.41277698[source]▶
> For some more exotic testing, I was able to run my own magic wormhole relay[1], which let me tweak some things for faster/more reliable huge file copies.

The lack of improvement in these tools is pretty devastating. There was a flurry of activity around PAKEs like 6 years ago now, but we're still missing:

* reliable hole punching so you don't need a slow relay server

* multiple simultaneous TCP streams (or a carefully designed UDP protocol) to get large amounts of data through long fat pipes quickly

Last time I tried using a Wormhole to transmit a large amount of data, I was limited to 20 MB/sec thanks to the bandwidth-delay product. I ended up using plain old http, with aria2c and multiple streams I maxed out a 1 Gbps line.

IMO there's no reason why PAKE tools shouldn't have completely displaced over-complicated stuff like Globus (proprietary) for long distance transfer of huge data, but here we are stuck in the past.

replies(4): >>41278538 #>>41279150 #>>41279898 #>>41284508 #
croemer ◴[18 Aug 24 00:19 UTC] No.41279150[source]▶
20MB/sec is 160Mbps, so wormhole wasn't that far off the 1Gbps. Sure not maxing out but within a factor of 6.
replies(3): >>41279692 #>>41279911 #>>41282474 #
1. dgoldstein0 ◴[18 Aug 24 03:21 UTC] No.41279911[source]▶
^ found the astronomer
replies(1): >>41280526 #
2. anonymousiam ◴[18 Aug 24 06:23 UTC] No.41280526[source]▶
I (Electrical + Software Engineer) once worked for a physicist who believed that anything less than an order of magnitude was merely an engineering problem. He was usually correct.
replies(4): >>41280572 #>>41281479 #>>41282602 #>>41285264 #
3. vasco ◴[18 Aug 24 06:35 UTC] No.41280572[source]▶
I was taught the same. To not care a lot about things under an order of magnitude. Over the years when planning large software projects or assessing incidents and so on, the 1 order of magnitude threshold helped me often.
replies(1): >>41281098 #
4. croemer ◴[18 Aug 24 08:57 UTC] No.41281098{3}[source]▶
Bingo, I studied Physics!
5. elashri ◴[18 Aug 24 10:31 UTC] No.41281479[source]▶
As a physicist, I think this is correct too :). You don't start to see problems with things under that, unless they are deviation from standard model predictions.
6. sva_ ◴[18 Aug 24 14:31 UTC] No.41282602[source]▶
Variance in accuracy of this statement also safely within one order of magnitude
7. bscphil ◴[18 Aug 24 20:34 UTC] No.41285264[source]▶
An order of magnitude isn't a defined quantity, it depends on what base you're working in.
replies(1): >>41292709 #
8. newaccount74 ◴[19 Aug 24 17:03 UTC] No.41292709{3}[source]▶
The difference between log 2, ln, and log 10 is less than an order of magnitude, so to a physicist it's all the same :)