MacOS Catalina: Slow by Design?

Up until the release of Catalina, I've always upgraded to the latest version of macOS within a month or two. But some of the changes this time is really stopping me from upgrading.

As of Catalina, there's no sane way to install the Nix package manager without losing functionality because macOS now disallows creating new files in the root directory[1]. Nix stores its packages in the /nix directory and it's not possible to migrate without causing major disruptions for existing NixOS and other Linux users. This is too bad, since apart from Nix being a nice package manager, it also provides a sane binary package for Emacs. The Homebrew core/cask versions only provides a limited feature set[2][3].

[1]: https://github.com/NixOS/nix/issues/2925

[2]: https://github.com/Homebrew/homebrew-core/issues/31510

[3]: https://github.com/caldwell/build-emacs/search?q=support+is%...

It's funny, I just had to do this a few days ago.

This comment has worked for me on two machines: https://github.com/NixOS/nix/issues/2925#issuecomment-539570...

For me it's aperture. I like the interface better than lightroom, and I don't want to pay a monthly fee to have access to my photo library which I only add to once in a while. It's a shame because it's a great piece of software, and even the UI doesn't feel dated, but I just won't be able to run it if I upgrade.

There's just so many problems with that approach:

1. You have to create a separate volume just to install a package manager, which is a poor user experience

2. A separate volume means FileVault won't work out of the box

3. The volume can be mounted only after GUI apps are brought up

4. Restoring after sleep might fail because of 3

All of these are mentioned in the Github issue, but it might be hard to find because it requires so many clicks and scrolling to view the whole thread.

Might want to look at Capture1 at this point.

IMHO the original choice of the path seems incredibly ill-advised and the main burden lies with the original developers.

sometimes old errors and mistakes come back and bite

It only seems that way now because some platforms have begun locking down their root directories. Nix, by design, doesn't conform to the FHS way of organizing directories so it made perfect sense to use /nix when the decision was originally made.

What should the default nix store path have been then?

1 — Sure. But Nix isn't exactly the most friendly package manager to begin with. I wouldn't recommend it if you're not comfortable creating volumes.

2 — Could you explain? Mine is on and working, I didn't need to do anything else.

3 — Is this if you have login items that need nix to be available? I don't have this so I haven't noticed.

4 — I've never run into this, but again I might just not use Nix for the kind of things that would cause issues.

I second this. Any tool which creates its own directory in the filesystem root (and cannot run from any other location) is inherently doing it wrong by any measure.

> Nix, by design, doesn't conform to the FHS way of organizing directories

That's why /opt/ exists. What's wrong with /opt/nix/ ? Or /var/opt/nix/ for read-write files that need not be a fixed part of any package installation (the Unix equivalent of system-wide "Application Data").

Brew never had this problem because they chose a sane path without corrupting the system directory. It’s a bad design on part of NixOS and one can even say the changes in the macOS were designed to encourage good/sane design.

What is special about /nix that would make it better suited elsewhere? Aesthetic? Clutter? I don't think there are any technical reasons why the root of the filesystem is important. The /nix folder is just another folder with some ACLs/Permissions (however OSX works, idk)

In my very limited (I don't use nix) opinion, the default of /nix isn't an issue, but rather:

> and it's not possible to migrate without causing major disruptions for existing NixOS and other Linux users.

Software that can't be re-parented without breaking is destined to create problems for users... eventually.

> Nix, by design, doesn't conform to the FHS way of organizing directories so it made perfect sense to use /nix when the decision was originally made.

Refusing to conform to the FHS doesn't mean their decision made sense; refusing to conform to the FHS means they made a bad decision in the past and everything progressed from there.

It doesn't 'seem that way now because some platforms have begun locking down their root directories'; it seems that way because creating arbitrary directories in / is a terrible idea, and has been at least since I started using UNIX/Linux systems in the 90's.

Fact is, they made a bad design choice, and now it's come back to bite them (and their users) in the ass.

If you truly want to be "cross-platform" with long-term future proofing in mind, `/nix` is (edit: was) probably the most stable choice.

I get it, people are sensitive about the root directory. "But it's where ALL the stuff lives!". So yeah, try not to ever run 'rm -rf /' (even though this is blocked in most cases now).

But why make it completely inaccessible for creating files/directories in? So much hand-holding for people to make it impossible for a user to ever make a mistake just locks down the ecosystem more, forcing developers to implement proprietary hacks that don't scale properly.

`/var/opt/nix` and `/opt/nix` are options, sure. But you cannot guarantee that those directories will exist on every platform. And if you have to create them, why is this better than `/nix`?

The obvious option would be /opt/nix, /usr/local/nix, or something to that effect. /nix is a clearly obviously bad choice, and now we're starting to see why.

Exactly. What's more, if we're talking about user hostility, how hostile is when a software doesn't provide a configurable install dir? It's literally a single damn variable!!

For what it's worth, Aperture, iPhoto and iTunes can be made to run in Catalina. People figured out last year what hacks were needed and there is a tool called Retroactive that will automate the steps:

https://github.com/cormiertyshawn895/Retroactive

Got some discussion on HN [1] about 3 months ago amongst other places, cool bit of sleuthing in the vein of efforts to get versions of macOS running on Macs older than officially supported. Personally I'm somewhat resigned to needing VMs to run certain older software, with a big one for me being Creative Suite CS6. Like you I have no interesting in buying into Adobe's subscription lock-in. But it's nice that some stuff can keep running without that layer for a while longer. Hopefully it'll still be possible in 10.16.

----

1: https://news.ycombinator.com/item?id=22454069

/usr/local/something

This is not the case. The problem is that caching is based on the default path which is /nix. So they would have to rebuild all caches.

What it was: `/nix` Or maybe `/notroot/nix` to make people happy.

"The root directory is untouchable" is a new fear-based imperative that would have been hard to predict.

Maybe they shouldn't have built it that way then. In my experience nix is nothing but a huge pain in the ass if you don't buy fully into the system, weird design decisions and all

Nix isn't designed as an application. It's designed as a system package manager.

> creating arbitrary directories in / is a terrible idea, and has been at least since I started using UNIX/Linux systems in the 90's

Why?

> doesn't provide a configurable install dir

This is completely false. You can change the installation directory at the cost of losing binary packages. When you change it, packages would be built from source instead. This is what Homebrew does too.

What's more, I don't think many package managers provide this option. Not apt, not yum.

Writing file to /nix shouldn't corrupt the system directory either. What exactly do you mean by "bad design"?

It's also an application, it just happens to manage other applications

There's a fix tool/hack to run Aperture on Catalina, called Retroactive.

https://github.com/cormiertyshawn895/Retroactive

It also works for iTunes and iPhoto. Sadly it won't fix any of the other known Catalina issues, of course! ;)

Nix living at a predefined path is integral to how it works. An executable does not dynamically link to a generic "ncurses" but (via rpath) links to a specific compiled version of ncurses (such as /nix/store/81rb87agmp9cbsvg2xm2n4kp9c6309lv-ncurses-6.2). This is the root of all the benefits of Nix such as being able to install things side-by-side that use different versions of things or upgrade and rollback without problems.

That predefined path being the same (/nix) across all users of nixpkgs is required to be able to share binary packages (you could perhaps build everything from source, but that's a lot of time, more time even than something like gentoo because package updates require all dependencies to be rebuilt as well).

You can call it an insane choice or bad design, but there aren't a whole lot of options here. Could Nix move to a different path? Maybe, but is there a path that all operating systems could abide? If the new path stops working in some future OS, will it still be insane and bad design? Again, maybe, but I happen to love Nix and I use is on macos because it makes my life easier (and I'm on macos for work reasons). I'm willing to bend and do a lot of legwork to be able use Nix, and I'm upset with the Catalina situation.

Can follow some discussion here https://github.com/NixOS/nix/issues/2925

Not conforming to the FHS is what makes Nix possible. You won't get Nix's reproducibility without it.

Historically / has been reserved for the use of the Unix system (the distribution that packaged it, not the computer you're running on). Local programs were installed to /usr/local. Packages installing themselves in /packagename are making your root directory like Windows' Start Menu. Furthermore, if your, say, Physics department has 20 machines, your sysadmin would install everything on an NFS share, which probably got mounted at /opt. Your sysadmin definitely did not want to mount /this, /that, /theother.

So while /nix is no problem from the filesystem driver, it is completely flaunting established Unix norms.

The UI is way worse than either Aperture or Lightroom, but the editing is powerful, and you can download the full version for free if you have a Fuji or Sony camera, IIRC.

> Brew never had this problem because they chose a sane path

How so? Taking over /usr/local as Homebrew does is guaranteed to cause conflict. Using a dedicated file hierarchy as Nix does is quite reasonable and there's nothing magical about rooting it at /.

Because the root directory might be on a very small partition (perhaps only a few hundred megabytes), while other mount points like /usr might have more space; the only things which should be in / are the things which are necessary to mount the other filesystems (perhaps through the network using NFS).

(Yes, nowadays hard disks are much larger, we have things like initrd, and we now make /bin and /sbin symlinks to within /usr, but the parent comment did mention the 90s...)

It's not that installing Nix is impossible on macOS, it's just that it has some hard-to-ignore limitations now.

1. Having to create a volume when a plain old directory should suffice is insane. It's creating a hassle for no good reason for users.

2. /nix would be unencrypted by default if kept in a separate volume. There's also the problem of how to unlock it upon boot.

3. Login items is a very common use case so not supporting it would be problematic for many users.

4. Unreliable sleep is an even bigger problem.

I'm probably missing something, and please let me know if so and why, but it sounds like a chroot could solve path reproducibility.

> Brew never had this problem because they chose a sane path without corrupting the system directory.

Ha, no. They did the absolute worst thing they could have done and now that they are popular they think they "own" /usr/local. (They used to camp out in /usr, but Apple rightfully put a stop to that real quick when SIP came out.)

Homebrew itself recommends you not do this, and while it is getting better at working in this case you will still run into issues if you try to do certain things.

Can you explain the reasoning here? I can see it being _easier_ than doing it the right way but have trouble coming up with a scenario where it makes it _impossible_.

I believe Nix actually picks a volume so that it can be encrypted, and it uses one of the many ways to run a script before login (some of which still happen to work) to decrypt it?

This is why, of the two, I prefer Macports.

To be more clear, it wasn't designed as a third-party package manager. It's supposed to be part of the system.

Happy MacPorts user of just over a year as well, for a variety of reasons I won't get into here but that being one of them.

It's still problematic because that can only happen late in the login process.

I read that thread a couple weeks back (was doing some firmlink research and stumbled upon it) and I seem to recall someone there finding something that ran pretty early. Perhaps I'm misremembering? I am sure there is at least one way to get this done, but I'll have to go look into what it is.

If you have to `mkdir /nix`, what's wrong with `mkdir -p /opt/nix`? I don't see how one is "more stable" than the other. The big difference between the two is the later conforms to convention while the former doesn't.

[Spack](https://spack.io) uses patchelf and additional tooling to relocate it's binary packages to other paths. It generally works, although one has to special case things that burn their install directory into their builds (e.g. Perl).

Unix OS variants have pretty standard paths like /opt or /usr.

Going with /nix was basically the best way to run into trouble.

It's not really a desirable feature, but a limitation of the tools it has to work with, where e.g. specifying an rpath of $NIXROOT/store is not possible.

it's a single variable which many parts of the system need to have knowledge about, some parts which have basically no way to feed in a variable. You can change the root directory in nix, but that invalidates all binary packages, in part because rpath is not at all configurable.

For a modern, subscription-less alternative to CS6 look at serif's affinity suite (no direct lightroom equivalent there though)

Nix requires that each package only writes to a dedicated directory in /nix/store. For example, files for Firefox 33.1 package would go into /nix/store/b6gvzjyb2pg0kjfwrjmg1vfhh54ad73z-firefox-33.1. By not dumping files from every package in a common directory such as /usr, it requires each package to be explicit with its dependencies. This allows for many nice things explained elsewhere (e.g., https://nixos.org/nix/).

Very satisfied MacPorts user since 16 years. I really don’t get why brew is a thing...

Unfortunately, what you're asking for is fundamentally impossible with binary package managers.

Because 10 years ago when I first started installing software from the command line on my mac, a large number of the packages I wanted to install were very outdated on MacPorts, and {CURRENT_VERSION} on homebrew.

Also `brew cask` is nice.

EDIT: also that Macbook had a 128GB HDD, so space was kinda precious, and MacPorts installing its own version of libs that were already on the system was literally taking up GBs of space.

`mkdir -p /opt/nix` assumes that there is a convention, and that this is the correct convention - which may not be the case for every situation, and would result in creating unnecessary nested directories.

You could make a more sophisticated installation script that attempts to install Nix into conventional locations depending on the specific operating system - or user input - but if you want a simple catch-all, simple installation script `/nix` was a perfect cross-platform installation location, until now.

How does it "take over" /usr/local? You can still `./configure --prefix=/usr/local` on your own software and things continue to work as long as you're not installing the same thing that brew is.

I still don't understand why that can't be solved by putting everything in /opt/nix/store

> Brew never had this problem because they chose a sane path without corrupting the system directory.

That's a hilarious assertion. Back in the days brew's takeover of /usr/local caused OSX upgrades to get stuck for hours on end (some folks reported more than 12h).

> How does it "take over" /usr/local?

Because it shoves all its shit there without asking.

Macports actually did it correctly and IME never had any issue.

Or NIX_PATH, or ~/.nix, et c.

I am infinitely tired of this node_modules “we know better than you, it isn’t configurable and will never be configurable so stop asking” hubris. It’s not open source entitlement to say that a maintainer with that attitude is bad and wrong.

My homebrew is installed to ~/Library/Homebrew and while they claim it’s unsupported, it works, and if it stops working, then I’ll stop using Homebrew.

I don’t trust software that demands root when it doesn’t need it.

Packages can not 'accidentally' depend on other packages as the only way to depend on a package is by referring to their full path which your learn by evaluating that package.

If you have an application that calls /usr/bin/nginx but doesn't declare a dependency on nginx; but you had nginx installed already the package works fine and you only find our later

In nix you can't do this as you don't know nginx's path without defining a dependency on it; so you don't gain undeclared dependencies on accident.

By forcing a different path you find these things at build time not at run time.

It’s because Nix was designed to be part of the OS, as integral as apt is for Debian installations. The ability for it to live side by side with another packaging system is just a side-effect of how it was designed, not part of the original goals.

You can create permanent symlinks inside / by creating a file called /etc/synthetic.conf - 'man synthetic.conf' has the full documentation. This sounds like it would solve the issue?

Installing several versions of the same piece of software is central to Nix.

While locking all needed versions for a specific application provides stability, I can't believe it doesn't come without a large increase of complexity, especially in connection security upgrades which triggers other libraries to need an update as well.

This is wrong. Everything not specified in the FHS is the domain of the administrator and is a contract with the OS about what directories it wont touch.

Nix, operating outside of the FHS, did the literal correct thing because there is no guarantee that the OS won’t install something in /opt/nix but there is a guarantee that it won’t touch /nix.

But yeh rooting everything under /usr/nix or /opt/nix would've probably been a better choice.

What annoys me more is that a popular nix Fork GNU Guix _did_ change the path; but they made the same mistake again (it's rooted under /gnu) whilst they already has the hindsight that a non-standard directory might be problematic

You can install Nix without losing functionality, it’s just annoying because it requires setting up a separate volume, and if you want it encrypted and available before the GUI session restores then you have to use a login script to force-mount it. Personally I just keep my Nix volume unencrypted because I don’t build any proprietary software in it and I don’t care if someone can see what I have installed.

I really wish Apple would give third parties the ability to create firmlinks (or at least give Nix one), or barring that, give us a sane way to mount encrypted volumes at the same time that the system volume is unlocked.

In addition to outdated ports, several times I had issues with macports mucking with or otherwise interfering with the system-bundled copies of things which was a real headache.

Seems like the ideal setup would be something like Homebrew, except it "lives" in the ~/Library/Brews/ folder or something to that effect.

Everything not specified in the FHS is reserved for use by the administrator. The FHS isn’t all-encompassing. It’s a contract about what directories the OS wont touch.

Generally you’re right and if you make a piece of software not follow the FHS you better have good reason. Nix, I think, makes a solid case since existing outside of the FHS is the only safe way to not conflict with every package manager.

It’s a capped version with some missing functionality (like layers), but it’s still a great piece of software.

The problem is that /opt/nix isn’t safe from the OS and Nix is explicitly software that doesn’t follow the FHS so it makes no sense to install it in a prefix.

/opt/local/nix is probably safe.

/usr/local is a prefix and contains local software that follows the FHS (i.e. libs in lib/, docs in doc/ binaries in bin/). Nix explicitly doesn’t do that so it would be inappropriate to install it there.

Homebrew does this far more often. MacPorts is off in its own world in /opt/local, which is actually mildly inconvenient sometimes because a lot of things won't pick it up when you want them to.

I think part of it is that they just advertised a lot more. When Homebrew came out, I seem to recall them advertising MacPorts as basically being old and busted. (Not literally those words, probably, but that was the general gist.)

Five or so years ago I evaluated between brew and macports, macports package were much more out of date while I needed fairly recent packages and brew had more of what I needed at the time.

> `mkdir -p /opt/nix` assumes that there is a convention

A correct assumption on virtually all relevant extant systems...

> which may not be the case for every situation

In the supposed scenario where the assumption isn't correct, the downside of /opt/nix vs /nix is basically insignificant. What's the overhead of one level of directory nesting, a single extra inode? Big whoop.

Also, Macports never phoned home to Google without asking permission or notification, unlike Homebrew.

It could have been /opt/nix and been compliant with FHS, and kept all the benefits you mention.

You can use an alternate path with Nix. When you choose to do that, you will have to build all packages from source instead of installing prebuilt binaries.

Hindsight is 20/20. It wasn't /opt/nix for reasons I do not know. In the context of NixOS, there's little reason to consider FHS. Only when using Nixpkgs outside of NixOS does the /nix choice look poor. I don't know which came first.

/opt/nix is FHS compliant and would work fine.

That's an interesting point. But it's not just rpaths, there are many references to things within the nix store. I suspect it would quite difficult to make them bound at runtime or something, but would be nice if possible.

ports are for old beards, brew for cool hipsters.

That makes sense, and is good news. I withdraw my complaint against Nix; in my defense my ignorance was based on the thread on their GitHub about how Catalina makes Nix basically unusable. Turns out those people were both a) wrong and b) speaking authoritatively from ignorance. :/

I’m quite glad I can just install it somewhere else, and finally ditch the Homebrew spyware. Thank you for letting me know!

And what is the issue with leaving it as `/nix`, which is (was) accessible on virtually all extant systems? Other than "the root folder is special!"

The Nix abides.

Why are you apologizing for Apple? I too have always had my own path in / (/u for my NFS mounted homes). I guess I just learned of yet another reason I will never go to Catalina (or buy any more macOS hardware).

Tiny typo that confused me, I think you mean:

> a lot of things won't pick it up when you want them to.

Thanks! I fixed it to limit confusion.

Easy solution: /nix as its own partition with plenty of space.

I'm much happier with their stance on it, too: https://lists.macports.org/pipermail/macports-dev/2019-March...

Completely off-topic, but do you somehow get notifications about replies to your comments? You often manage to respond within a couple minutes. :)

Please read https://news.ycombinator.com/newsguidelines.html and note the final two guidelines.

As a sysadmin you can do anything you want and are free to deal with the breakage you cause yourself. But the FHS pretty clearly establishes what behavior an application such as Nix should have: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03.html#p...

Quote:

"Applications must never create or require special files or subdirectories in the root directory. Other locations in the FHS hierarchy provide more than enough flexibility for any package."

They can choose not to abide the FHS, and that's fine if the users are happy with that tradeoff. But OSes observing the FHS are free to break Nix's expectations because they don't align with the FHS.

And NixOS is a great idea, but Nix advertises itself as "a powerful package manager for Linux and other Unix systems" in its the official site's description. Yet by not abiding to the FHS, they opened themselves up for breakage.

Nothing about that says that the path has to be /nix — it would work just as well with the standard layout under /opt/nix.

It also seems like it doesn't really help with the stated problem since a developer who would hard-code /usr/bin/nginx but not list it as a dependency would almost certainly just use whatever `which nginx` returns. The thing which would solve that is depending on a precise version or hash, and if you're doing that, the path prefix doesn't really matter.

Nothing was wrong with it, except that it violated convention. As I said: "The big difference between the two is the later conforms to convention while the former doesn't."

Violating convention comes with risk. Whether violating convention and assuming that risk is a good idea depends on whether the risk is worth the reward. For Nix, I don't think it was.

That’s an after the fact feature; my understanding is that the original design was intended to be the sole package manager on the system.

This is backed up by the fact that NixOS and Nix appear to have both been created at the same time; 2003.

Not sure about five years ago, but these days it's usually not to bad for popular packages.

You're not _wrong_, but I'm not sure that those reasons really mean anything. They were all new and arbitrary at some point and we're a long way from Unix.

> it is completely flaunting established Unix norms.

Also, _nix_ is completely flaunting established Unix norms in more ways than one. /nix is where all of the nix stores go, the immutable bundles that get pieced together to form all of the stuff you install. It could go in /var/nix or wherever, it doesn't really matter.

But putting it in /nix is kind of nice in that it's so different from the purposes of the rest of the filesystem. /nix doesn't behave like the rest of a normal Linux system, so it is separate. You can still symlink from /usr/local/bin/foo -> /nix/store/abcdef-1.1.0/bin/foo so the rest of your system has the same expectations.

But you’re missing the essential point which is that the FHS a set of rules that only apply to the distribution.

> Applications [shipped by the distribution] should not...

> Distributions should not create new directories in the root hierarchy without extremely careful consideration of the consequences including for application portability.

The OS is free to break 3rd party applications that don’t follow their rules but strict adherence to the FHS is not the justification for doing so.

That's not necessarily true because it ensures that you own an entire namespace separate from the OS install, which in Nix's case makes a lot of design sense given its use case(s).

Thanks for explaining! It sounds like I am just lucky with my set up not to run into issues. Hopefully they come up with a solution soon.

You've got to keep it mind that Nix was designed as a system package manager and its primary target is NixOS. As such, I'd imagine that it wasn't all that unreasonable for Nix to assume ownership over /nix when the decision was made back then. Since Nix doesn't organize files according to FHS recommendations, it's never going to be FHS compliant. If so, what benefits would there be for Nix to choose /opt/nix over /nix? After all, /opt is where third-party packages reside and hardly the right choice anyways.

The fact that Nix can be used as a third-party package manager outside of NixOS was a nice side-effect of its design choices. If Nix was designed from the start as a third-party package manager, it might have placed itself in /opt/nix. However, Nix was made years before some platforms started to lock down the root directory. It worked perfectly well with /nix before this happened.

So instead of asking why Nix is placing itself in /nix instead of following some guidelines for traditional unix distributions, I think we should be asking why platforms are disallowing this. If a platform is going to make this big a breaking change, it'd better have a very good reason to do so. I fail to see the reason beside subjective aesthetics.

By your own words, tool like apt, yum, and pacman would all be "doing it wrong." It's just wrong to blindly apply any rules without considering the various presumptions that justifies it. Specifically, the general advice of not creating directories in the filesystem root mainly applies to individual packages and is inadequate for system-level package managers.