←back to thread

741 points chirau | 2 comments | | HN request time: 0.437s | source
Show context
theLiminator ◴[] No.44358222[source]
uv and ruff are a great counterexample to all those people who say "never reinvent the wheel". Don't ever do it just for the sake of doing it, but if you have focused goals you can sometimes produce a product that's an order of magnitude better.
replies(7): >>44358352 #>>44358435 #>>44358583 #>>44358604 #>>44360352 #>>44361046 #>>44362201 #
eviks ◴[] No.44358352[source]
They didn't reinvent the wheel, "just" replaced all the wood with more durable materials to make it handle rotation at 10 times the speed
replies(2): >>44358809 #>>44360563 #
socalgal2 ◴[] No.44358809[source]
I'd be curious to know exactly what changed. Python -> Rust won't make network downloads faster nor file I/O faster. My naive guess is that all the speed comes from choosing better algorithms and/or parallelizing things. Not from Python vs Rust (though if it's hard to parallelize in Python and easy in rust that would certainly make a difference)
replies(7): >>44358971 #>>44359017 #>>44359201 #>>44359347 #>>44359610 #>>44359914 #>>44360074 #
1. captnswing ◴[] No.44359914[source]
Extremely interesting presentation from Charlie Marsh about all the optimizations https://youtu.be/gSKTfG1GXYQ?si=CTc2EwQptMmKxBwG
replies(1): >>44361968 #
2. socalgal2 ◴[] No.44361968[source]
Thanks. So from the video the biggest wins were

1. they way get the metadata for a package.

packages are in zip files. zip files have their TOC at the end. So, instead of downloading the entire zip they just get the end of the file, read the TOC, then from that download just the metadata part

I've written that code before for my own projects.

2. They cache the results of packages unzipped and then link into your environment

This means there's no files being copied on the 2nd install. Just links.

Both of those are huge time wins that would be possible in any language.

3. They store their metadata as a memory dump

So, on loading there is nothing to parse.

Admittedly this is hard (impossible?) in many languages. Certainly not possible in Python and JavaScript. You could load binary data but it won't be useful without copying it into native numbers/strings/ints/floats/doubles etc...

I've done this in game engines to reduce load times in C/C++ and to save memory.

It'd be interesting to write some benchmarks for the first 2. The 3rd is a win but I suspect the first 2 are 95% of the speedup.