←back to thread

A critique of package managers

(www.gingerbill.org)
109 points gingerBill | 2 comments | | HN request time: 0s | source
Show context
NoboruWataya ◴[] No.45168116[source]
I see this a lot with Rust where I will depend on one or two external crates for a simple application and then I am shocked to see dozens of dependencies being pulled in when I go to build. I actually think Cargo's support for feature gates and conditional compilation could in theory be a strong mitigation against this as crates can avoid pulling in dependencies unless you actually need a feature that relies on them, but in practice it doesn't seem to work that way as I often see these complaints about Rust.

I sympathise with the arguments but IMO laziness will always win out. If Rust didn't have Cargo to automate dependency hell, someone would create a third party script to fill the gap.

replies(3): >>45168261 #>>45168360 #>>45168912 #
cmrdporcupine ◴[] No.45168261[source]
It is an organizational not a technical problem.

When I worked at Google every single dependency was strictly vendored (and not in the mostly useless way that Cargo vendors things). There was generally only one version of a dep in the mono repo, and if you wanted something.. you generally got to own maintaining it, and you had to make sure it worked for every "customer" -- the giant CI system made sure that you knew if an upgrade would break things. And you reached out to stakeholders to manage the process. Giant trains of dependencies were not a thing. You can do that when you have seemingly infinite budget.

But technology can indeed make it worse. I love Rust, but I'm not a fan of the loose approach in Cargo and esp Crates.io, which seems to have pulled inspiration from NPM -- which I think is more of a negative than positive example. It's way too easy to make a mess. Crates.io is largely unmoderated, and its namespace is full of abandoned or lightly maintained projects.

It's quite easy to get away with a maze of giant transitive deps w/ Cargo because Rust by default links statically, so you don't usually end up in DLL hell. But just doing cargo tree on the average large Rust project is a little depressing -- to see how many separate versions of random number generators, SHA256, MD5, etc libs you end up with in a single linkage. It may not be the case that every single one is contributing to your binary size... but it's also kind of hard to know.

Understanding the blast radius of potential issues that come from unmoderated 3rd-party deps is I think something that many engineers have to learn the hard way. When they deal with a security vulnerability, or a fundamental incompatibility issue, or have to deal with build time and binary size explosions.

I wish there was a far more mature approach to this in our industry. The trend seems to be going in the opposite direction.

replies(1): >>45175157 #
1. zokier ◴[] No.45175157[source]
In many ways traditional Linux distros operate on similar model as I imagine googles monorepo. Both aim to this "globally consistent" dependency situation where you have one version of each library and you patch up things from upstream when they don't fit.

I feel we need more of these kinds of distros so you don't need to manage dependencies directly from upstream and deal with the integration effort yourself. What if we had a Rust disto following this same model, where there is only one version of each dep, some reasonable curation, and also you had nice clear release cycles? I feel that could real boon for the industry.

replies(1): >>45175389 #
2. cmrdporcupine ◴[] No.45175389[source]
Unfortunately I think it'd be too much putting the toothpaste back in the tube at this point. The way people are used to working in Rust is like filling their shopping cart with crate treats and then gluing them together.

I dunno maybe what is needed is a crates.io alternative that is highly highly moderated and highly highly selective. A subscription service with a paid staff that manages the packages and makes sure their deps are minimal, consistent with each other, secure, etc.

I can see that being a service that some corporations might pay for. I just came off a gig at a medical devices company that was using Rust and the software BoM side of things kept me up at night. The list of dependencies in the root workspace was long, and in my imagination, full of terrors.

Maybe they'll be fine, but it's not a practice I would recommend if I were starting such a project from scratch.