NPM debug and chalk packages compromised

https://socket.dev/blog/npm-author-qix-compromised-in-major-...

While it sucks that this happened, the good thing is that the ecosystem mobilized quickly. I think these sorts of incidents really show why package scanning is essential for securing open source package repositories.

Does the AI detect the obfuscation?

Either way, the parent post is clearly ambulance chasing rather than having a productive conversation, which should really be about whether or not automatically downloading and executing huge hierarchal trees of code is absolutely fucking crazy, rather than a blatant attempt to make money off an ongoing problem without actually solving anything.

In this incident, we detected the packages quickly, reported them, and they were taken down shortly after. Given how high profile the attack was we also published an analysis soon after, as did others in the ecosystem.

We try to be transparent with how Socket work. We've published the details of our systems in several papers, and I've also given a few talks on how our malware scanner works at various conferences:

* https://arxiv.org/html/2403.12196v2

* https://www.youtube.com/watch?v=cxJPiMwoIyY

You can call it ambulance chasing, but I think this is a good thing for the whole software ecosystem if people aren't accidentally bundling cryptostealers in their web apps.

And regarding not copying massive trees of untrusted dependencies: I am actually all for this! It's better to have fewer dependencies, but this is also not how software works today. Given the imperfect world we have, I think it's better to at least try to do something to detect and block malware than just complain about npm.

It seems strange to attack a service like this right after it actively helped keep people safe from malware. I'm sure its not perfect, but it sounds like they deserve to take a victory lap.

So just because a lock isn't 100% effective at keeping out criminals we shouldn't lock our doors?

That’s like lock companies parading around when their neighbour is murdered during a burglary but they weren’t because they bought a Foobar(tm) lock.

Seems good to me. I want more attention and more tooling around this problem. You seem mad at them for helping solve a real problem?

Very insightful.

Thanks for the links in your other comment, I'll take a look!

“Chat, I have reading comprehension problems. How do I fix it?”

You can't catch everything with normal static analysis either. LLM just produces some additional signal in this case, false negatives can be tolerated.

There are cases an LLM may be able to catch that their static analysis can't currently catch. Should they just completely ignore those scenarios, thereby doing the worst thing by their customers, just to stay purist?

What is the worst case scenario that you're envisioning from an LLM hallucinating in this use case? To me the worst case is that it might incorrectly flag a package as malicious, which given they do a human review anyway isn't the end of the world. On the flip side, you've got LLM catching cases not yet recognised by static analysis, that can then be accounted for in the future.

If they were just using an LLM, I might share similar concerns, but they're not.

when static analysis does it, it's called a "misclassification"