←back to thread

158 points kenjackson | 2 comments | | HN request time: 0s | source
Show context
roblabla ◴[] No.41031699[source]
This is some very poor journalism. The linux issues are so, so very different from the windows BSOD issue.

The redhat kernel panics were caused by a bug in the kernel ebpf implementation, likely a regression introduced by a rhel-specific patch. Blaming crowdstrike for this is stupid (just like blaming microsoft for the crowdstrike bsod is stupid).

For background, I also work on a product using eBPFs, and had kernel updates cause kernel panics in my eBPF probes.

In my case, the panic happened because the kernel decided to change an LSM hook interface, adding a new argument in front of the others. When the probe gets loaded, the kernel doesn’t typecheck the arguments, and so doesn’t realise the probe isn’t compatible with the new kernel. When the probe runs, shit happens and you end up with a kernel panic.

eBPF probes causing kernel panics are almost always indication of a kernel bug, not a bug in the ebpf vendor. There are exceptions of course (such as an ebpf denying access to a resource causing pid1 to crash). But they’re very few.

replies(4): >>41031896 #>>41032164 #>>41032610 #>>41034621 #
mbesto ◴[] No.41032164[source]
> just like blaming microsoft for the crowdstrike bsod is stupid

Wait, how is this stupid? Unless I'm missing something, wasn't the patch part of a Microsoft payload that included an update to Crowdstrike? Surely Crowdstrike is culpable, but that doesn't completely absolve Microsoft of any responsibility, as its their payload.

replies(7): >>41032197 #>>41032249 #>>41032287 #>>41032415 #>>41032517 #>>41032630 #>>41032666 #
PedroBatista ◴[] No.41032630[source]
The "only" thing Microsoft should be blame is for signing a driver updates itself by taking "configurations ( code )" from userspace and turning a blind eye to all these loopholes because they also know it's not practical for them to sign all the driver code that goes into the kernel.

Maybe most of these "drivers" shouldn't be in Ring0 to begin with? This is a general problem and the norm, Windows is just another OS that allows this this way.

replies(1): >>41033958 #
dathinab ◴[] No.41033958[source]
it's not that simple

user-land drivers are a thing, heck they are the standard for modern micro kernel architectures

and even with hybrid kernels pushing part of the code out of the kernel into something like "user land co-processes" is more then doable, now it's now trivial to retrofit in a performant way and flexible way but possible

Mac has somewhat done that (but I don't know the details).

On Linux it's also possible, through with BPF a bit of a in-between hybrid (leaving some parts of the drivers in kernel, but as BPF programs which are much less likely to cause such issues compared to "normal" drivers).

A good example for that is how graphic drivers have developed on Linux, with most code now being in a) the user-land part of the driver and b) on the GPU itself leaving the in kernel part to be mostly just memory management.

And the thing is Windows has not enforced such direction, or even pushed hard for it AFIK, and that is something you can very well blame then for. You in general should not have a complicated config file parser in a kernel driver, that's just a terrible idea, some would say negligent and Windows shouldn't have certified drivers like that. (But then given that CrowdStrike insists that it _must_ be loaded on start (outside of recovery mode) I guess it would still have hung all systems even if the parsing would have been outsourced because it can't start if it can't parse it's config).

replies(1): >>41034683 #
1. traject_ ◴[] No.41034683[source]
> And the thing is Windows has not enforced such direction, or even pushed hard for it AFIK, and that is something you can very well blame then for.

Even here it's pretty hard to blame them due to antitrust concerns. Just google the word Patchguard.

replies(1): >>41054612 #
2. dathinab ◴[] No.41054612[source]
Thats is misleading.

Falcon uses apis like eBPF when available/usable , they are not stupid if they can use something which is more secure and reliable why should they not use it.

E.g. they use it on Linux, even through they could have created a custom kernel module (idk. if they maybe also have a custom kernel module tbh.).

And pushing for something doesn't mean banning other things. E.g. they could certify "following best security practices" and not give it to vendors not using the more modern APIs, while they can't block drivers based on it with the right marketing customers of CrowdStrike wouldn't want to buy it without such cert.

I.e. while MS doesn't provide viable ways to get the functionality Falcon and similar need without kernel modules it indeed would be a bit ridiculous for them to ban such software, and as of yet the do not.