macOS unable to open any non-Apple application

You've got to be kidding me. When Apple's servers are down, all Macs worldwide start freezing randomly? My XCode is hanging during builds, is this why?

This code signing enforcement stuff has gone way too far. Heads should roll for this.

That's correct. AFAIK Catalina will check online for everything, even binaries you compile yourself.

Almost certainly so. Apple has built chains of certificate trust very deep into the OS, along with apparently an assumption that this particular revocation service check is reliable & fast enough to call out to the network a lot.

Wait what happens if you don't have an internet connection? Can Macs not be used offline any more, surely that's still a relatively common use case for a laptop even today in a lot of places?

My understanding is that if you're offline, it skips this check and everything works fine. The reason this is a big deal is that the problem's on their end, so you're not offline, so it keeps trying and waiting instead of just letting you skip the check.

wait what, how?

Unfortunately there’s not a way to differentiate “we’re online but Apple’s servers are having issues — probably fine” and “we’re online and something something is preventing us from talking to them — something nefarious might be happening.”

On iOS, after a period of disconnection "the phone won't let you turn it on again until it goes online": https://youtu.be/BW32yUEymvU?t=1212

This seems to explain why my Mac was nearly unusable after a reboot last week. Turns out bind crashed on my firewall leaving me with no DNS.

After I restarted it I could actually launch apps other than terminal again.

If you don't have a connection, it just doesn't do the check. If you have a crappy connection like many of our students, it takes forever to check. If the server is down, life just sucks and non-Apple programs don't open.

That still seems weird. Why does running unrecognized software become safe when you're off line?

The behavior documented there is on FIRST run of a new executable.

You can like that behavior or find it unacceptable, but the issue in OP is not that, it was applying to executables that had already been launched plenty of times on the machine.

Yes, can someone clarify this? What the hell is going on here?

I experienced this a couple of weeks ago. My wifi was up, but my internetprovider was down. My Macbook came to a halt. Nothing worked anymore. The whole machine was extremely slow. When the internetprovider came back up again, everything was fine again.

Local copy of whatever Apple is checking? Update that daily (on sign on or something). Not going to catch zero day type stuff, but better than making the laptop unusable.

I'm guessing this is to help trigger the wipe of stolen phones.

Oh man, imagining a DDOS to fail that over.

Imagine how many people would lost their productiveness, maybe not at the big corps or govt (I assume they use a version of mac that call somewhere else/don't). But very very many people.

It's a security theater

That sounds like it might just be a bug. At least, I wasn't able to find any information whatsoever on this phenomenon on Google.

It doesn't become safe when you're offline, it's just that you're no worse off than you were. OCSP is s a certificate revocation protocol. It's only used for disabling certificates which were issued in good faith but now need to be revoked. Suppose Apple signs application X, and the signature is good for a year. Six months later, Apple discovers that application X contains malware, so they revoke the certificate. However, your computer doesn't know about the revocation until it checks the OCSP server, which requires you to be online. If you're offline, it just skips the check; the certificate wasn't revoked yesterday, so it's probably fine today too. The bug is that if you're connected to a network but can't contact the OCSP server (either because the OCSP server is down, or because you're not connected to the internet) then OSX keeps trying to connect and becomes sluggish and/or unresponsive. This is how we know that it's a defect rather than a deliberate choice; if they had decided to make the OS non−functional unless connected to the internet they would have done a better job of it.

It wouldn't surprise me if they one day wanted to require you to be online 100% of the time so that you can't skip the OCSP checks on applications, but I don't think that would go over very well. Apple wouldn't even be the first to produce applications that refuse to work if there's no internet connection. If you don't like the thought that they might one day spring this on you, I recommend investigating Linux.

Thank you. Phrased perfectly.

It's an invasive restriction, cynically designed, poorly engineered and improperly managed, that impairs your ability to function.. masquerading as security.

macOS is my favorite OS, but I don't need to use it. I was so psyched reading about the new Macbooks, and I've had to walk all that excitement back now. I cannot invest in a computer that locks me out of my job if a cable gets cut by a maintenance crew in Cupertino.

I think the point is that that database is too large to store on a single machine which is why it has to be ad-hoc queried and cached. I mean it will have the signature of every program run on a Mac.

So you can't use a computer on an airgapped network? That seems counterproductive if the objective is security.

I don’t really want a giant hash table on my disk either.

If your computer is actually airgapped and has no networking interfaces configured, you won't have this issue.

If your computer is able to resolve DNS for ocsp.apple.com but to connection-timeout all traffic, yes, you could possibly reproduce today's issue.

There's a non-zero chance that this bug has caused at least one death.

I agree that it’s security theater and a suspect implementation, but I was playing a game of “let’s imagine why someone might do this...”—

I’m wondering, suppose it was designed this way because part of the goal is to prevent the spread of malware, the fastest means of which is an internet connected computer. In that event, the feature only intrudes when the computer, by virtue of it’s internet connection, is a member of the threat class.

So... plausible?

Mandatory OCSP is security theater? That’s a pretty bold claim.

Apple built the computer; I exchanged money for the computer; now I own the computer.

Apple does not own the computer.

If Apple wants to own the computer, they can pay me instead.

Mandatory OCSP that fails open when you're offline is security theater.

A Bloom Filter[1] could be used as a lighter alternative. You probably have at least one of those in your disk now.

[1]: https://en.wikipedia.org/wiki/Bloom_filter

Had the same thing earlier in the week as the isp was doing maintenance two nights in a row. 5+ seconds to start sublime and other really basic apps. Apple apps had no problem of course.

Remembering the notarization problems people were having months ago I did some tests and confirmed.

Now have little snitch installed again and my laptops going to be an Apple orphan. So I never noticed this problem today by virtue of it pissing me off 2 days before.

That's why notarized applications should be stapled too. The stapling "ticket" is embedded in the app bundle and allows macOS to perform an offline check.

Basically you'll get the usual GateKeeper window, but with a slightly different message, along the lines of "I can't check this binary in realtime but I trust the embedded notarization".

Today I was late to join a corporate conference call. It took like 5 mins to start conferencing software.

First time ever I'm genuinely frustrated with apple - macs are not those unicorn tools anymore that work reliable

Airgapped network — an IP LAN not connected to the internet. These do exist, sometimes permanently for security reasons, and sometimes just where external connectivity sucks but you still want your laptop to talk to your NAS.

OCSP fails open by definition because it is a revocation protocol. In the absence of revocation, a valid cert continues to be valid.

The problem here is simply that Apple did not build a short enough timeout into their client.

> Oh man, imagining a DDOS to fail that over.

That might be what we just saw happen.

If you are connected to a network without an Internet connection, it just becomes unusable. Internet connection is somewhat unreliable in my area, and I had an internet outage that lasted for days during the COVID lockdown. I feared it was a malware infection causing the slow down. I switched over to Linux not long after.

If you point the request at localhost, the problem resolves. This means that a cable getting cut in Cupertino won’t matter. It is a revocation protocol; it fails open.

The problem today is that not that the connection to the server failed, but that it succeeded very slowly. The result was an accidental denial of service on the client.

It is a bug, and an easily fixed one at that.

This particular issue is easy to work around for technical users; the _problem_ is the philosophy that made it possible.

This is the reason I can no longer use Apple computers - the continuous battle they are waging against the users freedom on all fronts - the anxiety of what they will do next to _my_ computer is too much.

Scary, but most likely true.

Agreed. These are really useful in various settings, but seem to be outside of most people's experience.

SelfDDOS. The first ever.

Might as well get a chromebook then hahaha

Are you referring to Steve Bannon who said Dr. Fauci should be beheaded? Or something else?

Good luck finding a suitable replacement. Microsoft does unpredictable things to Windows. Linux maintainers do unpredictable things to all sorts of things.

Your only real recourse is to compile everything from source after a thorough review every time...

...or else trust someone.

Sure Apple had a problem here, but there are so many other reasons to trust them over any other org that I can't in good conscience switch platforms, because there's so much more anxiety elsewhere.

They own the software.

You didn't pay for that. You licensed it from them.

> Linux maintainers do unpredictable things to all sorts of things.

With Linux you don't have to worry about every program you launch being reported to the mothership, or that failure of the mothership to respond would cause your computer to not function.

If you're not reading all the source of everything you're running, any or all of it it absolutely could be reporting usage/stats/your data to a "mothership".

Just because there's no single central org involved doesn't mean there aren't risks.

Microsoft Windows also uploads your private exe’s, and then runs them on Microsoft servers:

https://medium.com/sensorfu/how-my-application-ran-away-and-...

I'm going to make a bold claim but Linus made a claim to this effect. Security is important but it cannot be the only main priority when designing systems. Apple's mistake here is probably the main story but more generally this attitude (letting systems spectacularly fail for the sake of hypothetical security) is foolish and results in rather terrible bugs like this.

You need to set up your own DNS caching resolver and start selectively filtering out Apple domains. Pihole does that wonderfully. Ask your Apple geniuses whether they would help you setting it to make your Macs work.

Or defense in depth.

I hate it too, but 'theater' implies it isn't useful in any way.

Holy Shit. That should be illegal. All it needs is one rogue employee to potentially steal trade secrets? And dont tell me MS employees never go rogue after the recent events...

You don't need to read it, you just need to be able to read it.

Just because there are risks doesn't mean the risks are meaningfully comparable.

Make OCSP fail locked and it would be a software imprisonment protocol instead.

Code signing is an okay thing as long as the signing identities don't get discriminated. Android has had code signing ever since it was released, but you always generated the certificate yourself, and the purpose was simply to stop someone else from making an apk with the same package id that would install over yours and gain access to its data.

The thing Apple does, on the other hand, with trusting themselves more than the user, is disgusting. I'm mostly libertarian, but if I ever become a president, this would be one of the first things I'd make illegal, right after shortening the copyright term to like 3 years.

And probably a ruse to amass application usage stats.

The point stands: if you allow a host to resolve ocsp.apple.com to an unresponsive (timeout) address, it might break macOS the same as today — whether by air gap, by firewall, or who knows what else.

Surely it's against copyright law

Give me, the owner of the computer, over the keystore for the root certificates I trust, and code signing is great.

> I'm mostly libertarian, but if I ever become a president, this would be one of the first things I'd make illegal, right after shortening the copyright term to like 3 years.

As a libertarian I can see the argument for getting rid of presumptive copyright (and tanking the US economy), but the government preventing people from entering into contracts that you don't like? That's just hypocritical.

Ken Thompson won a Turing Award for showing how that isn’t the case: http://users.ece.cmu.edu/~ganger/712.fall02/papers/p761-thom...

> but the government preventing people from entering into contracts that you don't like?

It's not that. Plain and simple: in an ideal world, more money shouldn't grant more power and immunity. Governments should disincentivize this growth into the sky by, for example, progressive taxation for companies. The world would be a better place if tech companies actually competed with each other by making better products, not trying their damnest to lock everyone into their walled gardens to earn even more money they have no clue what to do with. Currently, when choosing something like a computer or a phone, you just pick one that sucks the least. There's no healthy competition.

With Android is the same. I have an App Firewall on my Android phone and since then the standard Android gallery app does not work really anymore. A lot of things break, for ex. when I_ like to send a file with Threema, I have to go offline, choose the file and then go online again. Otherwise the file dialoge does freeze. It's just standard these days. Also a lot of things break, if you are just on a network without internet connection. Welcome in 2020.

A law is only dealing with the consequences, it's not prevention.

> Surely it's against copyright law

It almost certainly is, but

1. You have to know it's happening before you can do anything about it

2. If your "work" isn't registered with the copyright office, you're limited to actual damages, which are probably close to $0

That's a fair point that I hadn't considered, and I appreciate it. But I still feel like "ability to use your computer as a service" is not something I signed up for.

Because it is not yet illegal to operate a computing machine that is not centrally monitored. New Normal, get used to it. Soon, this corner case will go away.

"Why were you offline when using your computer?"

[deleted]

TL;DR: It's an option that can be disabled, unlike on Mac. Also doesn't lock up your PC if Apple's network is having a bad day.

That does not sound like a libertarian view at all.

Right. The recent problem (in top-level OP, and that you were presumably experiencing) was not just first run, but the behavior explained at the GP link (https://news.ycombinator.com/item?id=23281564 , HN thread for https://lapcatsoftware.com/articles/catalina-executables.htm...) is just about first-run, so the behavior explained at the GP link is not sufficient explanation for the recent problem, it's not talking about the same thing.

Is this how we look for the next Stuxnet?

Often when I would see this type of error it would be when something silently drops TCP packets (rather than sending a RST). This is one way to configure a firewall, and it's indistinguishable from high latency. Hence the difference in behavior. If the address was unroutable, or immediately closed the connection, it would fail quickly (and presumably for the OCSP check, it would be skipped immediately). But when packets are silently dropped, it's up to the client to decide how long to wait for an ACK, which might cause a hang.

I've seen an identical problem where Chrome would hang for minutes when loading sites, and it was because I was in a firewalled environment that was outright dropping packets to Chrome's OCSP server.

We already know that, by design, macOS will report back to the mothership. If things are working 100% correctly, Apple will collect what programs you run and when you do so.

Linux won't report to the mothership by design. If things work 100% correctly, you don't have to worry about some company knowing what programs you run and when.

I've already found a replacement, Debian stable + i3wm has been my happy place for the last 5 years. No unexpected behavior changes on update, just bug fixes, it does what I tell it, nothing crazy like Debian maintainers dictating what binaries I can run... if you want more or less control you've got plenty of Ubuntu style distros in one direction and Arch style in the other.

If you're a media person then yeah, I feel bad for you, i've been there and it sucks, you're stuck with mac and windows if you require mainstream design apps.

May I direct your attention to https://reproducible-builds.org/

Plausible a la NSA, yeah?

I presume this setup wasn't public knowledge.

Libertarian is not a well defined word. I have a friend who identifies as a Socialist and a Libertarian. He believes that true libertarianism (anarchy) would result in a collapse of capitalism since there would be no state to enforce private property rights.

So yeah, always gotta find out what a person means when they say "Libertarian"

That what isn't the case? Pointing out additional threat vectors doesn't in any way contradict my point.

Funny how DNS has that same issue, and yet, we still decentralized it to a point, even if there is some inertia going on to keep it as centralized as possible.