CrowdStrike's Falcon Sensor also linked to Linux kernel panics and crashes

Back in The Good Old Days, an OS vendor would release a beta version and software vendors would test against it and fix problems before the stable OS version was released.

Obviously OS updates come out a lot more often these days than they used to - but we're also better at test automation than ever before, and beta software is easier to get than ever.

It sure would be nice if companies that decide to produce kernel modules and to support certain OSes could test those kernel modules against those OSes at the beta stage.

1.An eBPF probe is not a kernel module. An eBPF probe should never cause kernel panics.

2. RHEL didn't provide beta kernels before very recently, as far as I can tell.

3. Even if you caught an error then, you're still at the mercy of RHEL to fix it. If RHEL breaks a feature, you report it to them, and they decide to ship anyways... well, your product will still kpanic. I'm not talking hypotheticals: I haven't seen RHEL do that, but I've seen other distros do it.

> and they decide to ship anyways... well, your product will still kpanic

But then you are in position to share your customers that this will happen before it actually does and they can choose their way of proceeding.

One such way would be being careful with the update and then exercising their own support contracts with RH.

Emphasis added:

> An eBPF probe should never cause kernel panics.

Should, but did. This is the point at which to learn and adapt.

Also, kernels are software just like nearly everything else, and software is buggy. It's a balance obviously, but some basic defensive development can be a real savior for your users.

I don't know the details about this CrowdStrike incident, but I would also be surprised if you couldn't write an automated test (even a "smoke test") to quickly test out these new kernels before they hit your customers. Given what happened, it seems like negligence not to do that.

It's possible CS can do better, of course. But it's just wrong to blame them for the Linux crashes - they're not the ones that introduced buggy code and broke their users. RHEL/Linux did.

> An eBPF probe is not a kernel module.

But if it runs on the same privilege level as the rest of the kernel, then isn't it, for the purposes of this discussion, in effect "a kernel module"?

it doesn't—the semantics of ebpf confine it—so it isn't

Either

1) Those Crowdstrike unit files aren't ebpf probes, so the whole subject of ebpf probes is irrelevant here; or

2) They're obviously able to stop the rest of the kernel from even booting up (as Crowdstrike so convincingly demonstrated millions of times over[1]), so yes, they do indeed have at least as much power as any other bit of the kernel.

Either way, hunting around for nits to pick is a bit pathetic.

[1]: In July 0000002024...

denial of service is not the same thing as arbitrary code execution, and that goes double in kernel mode, but yes, it does seem that the linux implementation of ebpf had buggy sandboxing; i don't think allowing clownstrike to prevent booting was part of the intended objective

i wasn't hunting around for nits to pick; i was hunting around to see if you'd ever contributed any useful comments to the site. instead i found you making authoritative pronouncements about ebpf that were so wrong that you had evidently never read so much as a one-line summary of what ebpf was for. do you have a more promising historical comment to offer? perhaps something where people complimented your contribution as being informative?

have you ever made a worthwhile comment on hn?

on thursday, wahern posted this comment https://news.ycombinator.com/item?id=41061179 where they traced through the illumos/opensolaris source code to track down how a peculiar solaris interprocess communication mechanism worked, an investigation i had started but gotten stuck on. why can't you make comments like that instead of harassing me about how i format my comments?

the reason i'm asking is because i'd like to be able to talk to more people like wahern, but most of them avoid this site. a major reason why is that comments here frequently receive vacuous, aggressive responses like the comment you made the day before in https://news.ycombinator.com/item?id=41056718, where you launched a personal attack on me because you didn't like how i was formatting my comments

i'd like you to ⓐ apologize for doing that (this is not the first time you've done that to me personally; so far i haven't looked through your comment history far enough to find out how many other people you have a history of repeatedly harassing) and ⓑ commit to not doing it again

because i'm sure you're capable of making comments that make the site better instead of worse

> do you have a more promising historical comment to offer? perhaps something where people complimented your contribution as being informative?

> have you ever made a worthwhile comment on hn?

I might answer that. If I thought you were owed any justifications from me. Which I don't.

And no, I'm neither “harassing” you nor being “vacuous, aggressive”. This isn't ad hominem, it's ad habitem. You write here for other people to read, and I'd even appreciate many of your comments -- if they weren't so infuriatingly idiosyncratically formatted as to disrupt fluent reading. Have the fucking courtesy to write like a normal person, and you'll be treated like a normal person. To begin with, get the shift key on your keyboard unstuck so you can start your sentences with capitals. And in case your dot / period / full-stop key is totally gone, copy-paste some of these: ........... So you can end them properly too.

Because I'm sure you're capable of making comments without coming off like an illiterate buffoon.

And yes, BTW, you totally were. Careful now, you don't want to end up like chockablock again, do you?