←back to thread

392 points mfiguiere | 4 comments | | HN request time: 0.862s | source
Show context
RcouF1uZ4gsC ◴[] No.35470953[source]
> Buck2 is an extensible and performant build system written in Rust

I really appreciate tooling that is written in Rust or Go that produce single binaries with minimal runtime dependencies.

Getting tooling written in for example Python to run reliably can be an exercise in frustration due to runtime environmental dependencies.

replies(3): >>35471099 #>>35471103 #>>35471569 #
rektide ◴[] No.35471103[source]
Personally it seems like a huge waste of memory to me. It's the electron of the backend. It's absolutely done for convenience & simplicity, with good cause after the pain we have endured. But every single binary bringing the whole universe of libraries with it offends.

Why have an OS at all if every program is just going to package everything it needs?

It feels like we cheapened out. Rather than get good & figure out how to manage things well, rather than driver harder, we're bunting the problem. It sucks & it's lo-fi & a huge waste of resources.

replies(7): >>35471298 #>>35471361 #>>35471433 #>>35471640 #>>35471685 #>>35472010 #>>35472197 #
bogwog ◴[] No.35471640[source]
I don't think that matters so much. For building a system, you definitely need dynamic linking, but end user apps being as self contained as possible is good for developers, users, and system maintainers (who don't have to worry about breaking apps). As long as it doesn't get out of hand, a few dozen MBs even is a small price to pay IMO for the compatibility benefits.

As a long time Linux desktop user, I appreciate any efforts to improve compatibility between distros. Since Linux isn't actually an operating system, successfully running software built for Ubuntu on a Fedora box, for example, is entirely based on luck.

replies(1): >>35471780 #
rektide ◴[] No.35471780[source]
There's also the issue that if a library has a vulnerability, you are now reliant on every static binary updating with the fix & releasing a new version.

Where-as with the conventional dynamic library world one would just update openssl or whomever & keep going. Or if someone wanted to shim in an alternate but compatible library, one could. I personally never saw the binary compatibility issue as very big, and generally felt like there was a while where folks were getting good at packaging apps for each OS, making extra repos, that we've lost. So it seems predominantly to me like downsides, that we sell ourselves on, based off of outsized/overrepresented fear & negativity.

replies(1): >>35472269 #
preseinger ◴[] No.35472269[source]
the optimization you describe here is not valuable enough to offset the value provided by statically linked applications

the computational model of a fleet of long-lived servers, which receive host/OS updates at one cadence, and serve applications that are deployed at a different cadence, is at this point a niche use case, basically anachronistic, and going away

applications are the things that matter, they provide the value, the OS and even shared libraries are really optimizations, details, that don't really make sense any more

the unit of maintenance is not a host, or a specific library, it's an application

vulnerabilities affect applications, if there is a vulnerability in some library that's used by a bunch of my applications then it's expected that i will need to re-deploy updated versions of those applications, this is not difficult, i am re-deploying updated versions of my applications all the time, because that is my deployment model

replies(2): >>35472452 #>>35475496 #
rektide ◴[] No.35472452[source]
Free software has a use beyond industrial software containers. I don't think most folks developing on Linux laptops agree with your narrow conception of software.

Beyond app delivery there's dozens of different utils folks rely on in their day to day. The new statically compiled world requiring each of these to be well maintained & promptly updated feels like an obvious regression.

replies(2): >>35472926 #>>35472959 #
howinteresting ◴[] No.35472926[source]
Again, there is no alternative. Dynamic linking is an artifact of an antiquated 70s-era programming language. It simply does not and cannot work with modern language features like monomorphization.

Linux distros are thankfully moving towards embracing static linking, rather than putting their heads in the sand and pretending that dynamic linking isn't on its last legs.

replies(1): >>35473824 #
PaulDavisThe1st ◴[] No.35473824[source]
Whoa, strong opinions.

Dynamic linking on *nix has nothing to do with 70s era programming languages.

Did you consider the possibility that the incompatibility between monomorphization (possibly the dumbest term in all of programming) and dynamic linking should perhaps saying something about monomorphization, instead?

replies(1): >>35474200 #
howinteresting ◴[] No.35474200[source]
> Dynamic linking on *nix has nothing to do with 70s era programming languages.

Given that dynamic linking as a concept came out of the C world, it has everything to do with them.

> Did you consider the possibility that the incompatibility between monomorphization (possibly the dumbest term in all of programming) and dynamic linking should perhaps saying something about monomorphization, instead?

Yes, I considered that possibility.

replies(1): >>35474351 #
PaulDavisThe1st ◴[] No.35474351[source]
The design of dynamic linking on most *nix-ish systems today comes from SunOS in 1988, and doesn't have much to do with C at all other than requiring both the compiler and assembler to know about position-independent code.

What elements of dynamic linking do you see as being connected to "70s era programming languages"?

> Yes, I considered that possibility.

Then I would urge you to reconsider.

replies(2): >>35475191 #>>35479652 #
preseinger ◴[] No.35475191[source]
dynamic linking is an optimization that is no longer necessary

there is no practical downside to a program including all of its dependencies, when evaluated against the alternative of those dependencies being determined at runtime and based on arbitrary state of the host system

monomorphization is good, not bad

the contents of /usr/lib/whatever should not impact the success or failure of executing a given program

replies(1): >>35475419 #
PaulDavisThe1st ◴[] No.35475419[source]
Dynamic linking wasn't an optimization (or at least, it certainly wasn't just an optimization). It allows for things like smaller executable sizes, more shared code in memory, and synchronized security updates. You can, if you want, try the approach of "if you have 384GB of RAM, you don't need to care about these things", and in that sense you're on quicksand with the "just an optimization". Yes, the benefits of sharing library code in memory are reduced by increasing system RAM, but we're seeing from a growing chorus of both developers and users, the "oh, forget all that stupid stuff, we've got bigger faster computers now" isn't going so well.

There's also the problem that dynamic loading relies on almost all the same mechanisms as dynamic linking, so you can't get rid of those mechanisms just because your main build process used static linking.

replies(1): >>35476228 #
preseinger ◴[] No.35476228[source]
it allows for all of the things you list, yes, but those things just aren't really valuable compared to the reliable execution of a specific binary, regardless of any specific shared library that may be installed on a host

smaller executable sizes, shared code in memory, synchronized security updates, are all basically value-zero, in any modern infrastructure

there is no "growing chorus" of developers or users saying otherwise, it is in fact precisely the opposite, statically linked binaries are going extremely well, they are very clearly the future

replies(2): >>35476415 #>>35478228 #
bscphil ◴[] No.35478228[source]
> it allows for all of the things you list, yes, but those things just aren't really valuable compared to the reliable execution of a specific binary

> smaller executable sizes, shared code in memory, synchronized security updates, are all basically value-zero, in any modern infrastructure

This highlights the fact that you're extremely focused on one particular model of development, one where a single person or group deploys software that they are responsible for running and maintaining - often software that they've written themselves.

This is, obviously, an extremely appropriate paradigm for the enterprise. Static linking makes a lot of sense here. Python's virtual environments are basically the approved workaround for the fact that Python was built for systems that are not statically linked, and I cherish it for exactly that reason. Use Go on your servers - I do myself! But that doesn't mean it's appropriate everywhere.

Sometimes developers in this mindset forget there's a whole other world out there, a world of personal computers, that each have hundreds or thousands of applications installed. Applications on these systems are not deployed, they are installed. The mechanism by which this happens (on Linux) is via distributions and maintainers, and dynamic linking needs to be understood as designed for that ecosystem. Linux operating systems are built around making things simple, reliable, and secure for collections of software that are built and distributed by maintainers.

I'm firmly on the side of the fence that says that dynamic linking is the correct way to do that. All the benefits you mention are just a free bonus, of course, but I care about them as well. Smaller executable sizes? Huge win on my 256 GB SSD. Synchronized security updates? Of course I care about those as an end user!

replies(3): >>35478290 #>>35478304 #>>35488025 #
preseinger ◴[] No.35478290[source]
> Sometimes developers in this mindset forget there's a whole other world out there, a world of personal computers, that each have hundreds or thousands of applications installed.

it's not that i forget about these use cases, it's that i don't really consider them relevant

tooling that supports industrial use cases like mine is not really able to support end-user use cases like yours at the same time

linux operating systems may have at one point been built around making things as you describe by distribution maintainers, but that model is anachronistic and no longer useful to the overwhelming majority of its user base, the huge majority of software is neither built nor distributed by maintainers, it is built and distributed by private enterprises

replies(1): >>35486856 #
bscphil ◴[] No.35486856[source]
> > Sometimes developers in this mindset forget there's a whole other world out there, a world of personal computers, that each have hundreds or thousands of applications installed.

> it's not that i forget about these use cases, it's that i don't really consider them relevant

Yes, exactly! It's an extremely myopic vision. You've spent this long thread arguing against dynamic linking on the basis of what is only a small fraction of total human / computer interactions! By "not relevant" you mean not relevant to the enterprise. I grant that of course - but these uses cases are (by definition) relevant to hundreds of millions of PC users.

> the huge majority of software is neither built nor distributed by maintainers, it is built and distributed by private enterprises

The overwhelming majority of the software I run is built and distributed by maintainers. Literally, there are only a few exceptions, like static-built games that rarely or never change and are (unfortunately) closed source. I daresay that's true for the majority of Linux users - the vast majority of the software we install and use is not "built and distributed by private enterprises".

This reality is what Linux-on-the-desktop is built for. There are millions of people who are going to want to continue using computers this way, and people like me will continue contributing to and developing distributions for this use case, even if shipping static or closed-source binaries to Linux users becomes common.

replies(1): >>35494249 #
1. preseinger ◴[] No.35494249[source]
linux-on-the-desktop is also like statistically zero of linux installations (modulo mobile) but if that's counter to a belief of yours then we're definitely not going to make progress here so (shrug)

like i'm not sure you understand the scale of enterprise linux. a single organization of not-that-very-many people can easily create and destroy hundreds of millions of deployed systems every day, each with a novel configuration of installed software. i've seen it countless times.

replies(1): >>35498227 #
2. bscphil ◴[] No.35498227[source]
I think we're arguing on multiple fronts here and that is confusing things.

1. My point about Linux on the desktop is that there are in practice users like me who are already getting the (many) advantages of dynamic linking, and don't want to give up those advantages. To the point that some of us are going to support and work on distributions that continue the traditional Linux way in this area. In your view, the ecosystem has moved to software being built and distributed by private corporations. I don't think this has happened - on Windows software was always built and distributed this way; on (desktop) Linux it never was and largely still isn't!

2. My point about the desktop in general is that this use case matters to the vast majority of computer-using human beings much more than enterprise. The number of deployed containers that get created and destroyed every day doesn't change that fact, nor does the fact that Linux users are merely a tiny fraction of this desktop use case. This is what creates the myopia I was talking about - you're thinking about metrics like "number of systems deployed" whereas I'm thinking of number of human-computer interactions that are impacted. I don't think you can just discard what matters on the desktop or paint it as irrelevant. Desktop computing shouldn't be subordinate to the technical requirements of servers!

So to summarize the argument: (a) desktop use cases still matter because they comprise the majority of human-computer interactions, (b) dynamic linking and the maintainer model are the superior approach for desktop computing, and in fact complement each other in important ways, and (c) even if most desktop users can't take advantage of this model because of the dominance of closed source software and the corporate development model, desktop Linux can and does, and will hopefully continue to do so into the future.

replies(1): >>35499385 #
3. preseinger ◴[] No.35499385[source]
> Desktop computing shouldn't be subordinate to the technical requirements of servers!

i guess this is the crux of the discussion

linux desktop computing for sure _is_ subordinate to linux server computing, by any reasonable usage metric

i'm not trying to deny your experience in any way, nor suggest that dynamic linking goes away, or anything like that -- your use case is real, linux on the desktop is real, that use case isn't going away

but it is pretty clear at this point that linux on the server is wildly successful, linux on mobile is successful (for android), and that linux on the desktop is at best a niche use case

the majority of human interactions with linux occur via applications, services, tools, etc. that are served by linux servers, and not by software running on local machines like desktops or laptops

linux is a server operating system first and foremost

replies(1): >>35550153 #
4. rektide ◴[] No.35550153{3}[source]
whether we want unobservable ungovernable far off machines running the future forever, or whether we want a future where actual people can compute & see what happens seems to matter. the numbers may perhaps stack up to suborn PC needs to industrial computing needs now, but is that the future anyone should actually want? should the invisible hand of capital be the primary thing humanity should try to align to?

and where is the growth potential? is the industrial need going to become greatly newly empowered & helpful to this planet, to us? will it deliver & share the value potential out there? PC may be a smaller factor today, but i for one am incredibly fantastically excited to imagine a potential future 10 years from now where people start to PC again, albeit in a different way.

individual PCs have no chance. it's why the cloud has won. on-demand access wherever you are, consistent experience across devices is incredibly incredibly convenient. but networks of PCs that work well together is exciting, and we've only so very recently started emerging the capability to have nice easy to manage ops/automated multi-machine personal-computing. we've only recently emerged to maturity where a better, competitive personal computing is really conceivable.

it's been the alpha linux geeks learning how to compute and industrial players learning how to compute, and the invisible hand has been fat happy & plump from it, but imo there's such a huge potential here to re-open computing to persons, to create compelling interesting differently-capable sovereign/owned computing systems, that are free from so many of the small tatters & deprevations & enshittifications that cloud- that doing everyting on other people's computers as L-Users- unnerringly drops on us. we should & could be a more powerful, more technically-cultural culture, and i think we've severely underrated how much subtle progress there's been to make that a much less awful, specialized, painful, time-consuming, low-availability, disconnected effort than it used to be.