Most active commenters
  • zelphirkalt(4)
  • cluckindan(4)
  • lukan(3)

←back to thread

1208 points jamesberthoty | 56 comments | | HN request time: 1.361s | source | bottom
Show context
kelnos ◴[] No.45266878[source]
As a user of npm-hosted packages in my own projects, I'm not really sure what to do to protect myself. It's not feasible for me to audit every single one of my dependencies, and every one of my dependencies' dependencies, and so on. Even if I had the time to do that, I'm not a typescript/javascript expert, and I'm certain there are a lot of obfuscated things that an attacker could do that I wouldn't realize was embedded malware.

One thing I was thinking of was sort of a "delayed" mode to updating my own dependencies. The idea is that when I want to update my dependencies, instead of updating to the absolute latest version available of everything, it updates to versions that were released no more than some configurable amount of time ago. As a maintainer, I could decide that a package that's been out in the wild for at least 6 weeks is less likely to have unnoticed malware in it than one that was released just yesterday.

Obviously this is not a perfect fix, as there's no guarantee that the delay time I specify is enough for any particular package. And I'd want the tool to present me with options sometimes: e.g. if my current version of a dep has a vulnerability, and the fix for it came out a few days ago, I might choose to update to it (better eliminate the known vulnerability than refuse to update for fear of an unknown one) rather than wait until it's older than my threshold.

replies(35): >>45266995 #>>45267024 #>>45267360 #>>45267489 #>>45267600 #>>45267697 #>>45267722 #>>45267967 #>>45268218 #>>45268503 #>>45268654 #>>45268764 #>>45269143 #>>45269397 #>>45269398 #>>45269524 #>>45269799 #>>45269945 #>>45270082 #>>45270083 #>>45270420 #>>45270708 #>>45270917 #>>45270938 #>>45272063 #>>45272548 #>>45273074 #>>45273291 #>>45273321 #>>45273387 #>>45273513 #>>45273935 #>>45274324 #>>45275452 #>>45277692 #
gameman144 ◴[] No.45267024[source]
> It's not feasible for me to audit every single one of my dependencies, and every one of my dependencies' dependencies

I think this is a good argument for reducing your dependency count as much as possible, and keeping them to well-known and trustworthy (security-wise) creators.

"Not-invented-here" syndrome is counterproductive if you can trust all authors, but in an uncontrolled or unaudited ecosystem it's actually pretty sensible.

replies(8): >>45267054 #>>45267101 #>>45267444 #>>45268170 #>>45268880 #>>45270337 #>>45273381 #>>45273796 #
1. Ajedi32 ◴[] No.45267054[source]
If it's not feasible to audit every single dependency, it's probably even less feasible to rewrite every single dependency from scratch. Avoiding that duplicated work is precisely why we import dependencies in the first place.
replies(11): >>45267090 #>>45267094 #>>45267132 #>>45267222 #>>45267415 #>>45267471 #>>45268298 #>>45269164 #>>45270175 #>>45270363 #>>45270519 #
2. curtisf ◴[] No.45267090[source]
This is true to the extent that you actually _use_ all of the features of a dependency.

You only need to rewrite what you use, which for many (probably most) libraries will be 1% or less of it

replies(1): >>45267507 #
3. AlecBG ◴[] No.45267094[source]
Not sure I completely agree as you often use only a small part of a library
4. lukan ◴[] No.45267132[source]
"rewrite every single dependency from scratch"

No need to. But also no need to pull in a dependency that could be just a few lines of own (LLM generated) code.

replies(1): >>45267221 #
5. brianleb ◴[] No.45267221[source]
>>a few lines of own (LLM generated) code.

... and now you've switched the attack vector to a hostile LLM.

replies(3): >>45267528 #>>45269094 #>>45272158 #
6. gameman144 ◴[] No.45267222[source]
It isn't feasible to audit every line of every dependency, just as it's not possible to audit the full behavior of every employee that works at your company.

In both cases, the solution is similar: try to restrict access to vital systems only to those you trust,so that you have less need to audit their every move.

Your system administrators can access the server room, but the on-site barista can't. Your HTTP server is trusted enough to run in prod, but a color-formatting library isn't.

replies(1): >>45270529 #
7. zelphirkalt ◴[] No.45267415[source]
Most dependencies do much more than we need from them. Often it means we only need one or a few functions from them. This means one doesn't need to rewrite whole dependencies usually. Don't use dependencies for things you can trivially write yourself, and use them for cases where it would be too much work to write yourself.
replies(3): >>45267701 #>>45271035 #>>45271065 #
8. bennyg ◴[] No.45267471[source]
Sounds like the job for an LLM tool to extract what's actually used from appropriately-licensed OSS modules and paste directly into codebases.
replies(3): >>45267705 #>>45268191 #>>45269005 #
9. zahlman ◴[] No.45267507[source]
Indeed. About 26% of the disk space for a freshly-installed copy of pip 25.2 for Python 3.13 comes from https://pypi.org/project/rich/ (and its otherwise-unneeded dependency https://pypi.org/project/Pygments/), "a Python library for rich text and beautiful formatting in the terminal", hardly any of the features of which are relevant to pip. This is in spite of an apparent manual tree-shaking effort (mostly on Pygments) — a separate installed copy of rich+Pygments is larger than pip. But even with that attempt, for example, there are hundreds of kilobytes taken up for a single giant mapping of "friendly" string names to literally thousands of emoji.

Another 20% or more is https://pypi.org/project/requests/ and its dependencies — this is an extremely popular project despite that the standard library already provides the ability to make HTTPS connections (people just hate the API that much). One of requests' dependencies is certifi, which is basically just a .pem file in Python package form. The vendored requests has not seen any tree-shaking as far as I can tell.

This sort of thing is a big part of why I'll be able to make PAPER much smaller.

replies(1): >>45270855 #
10. zelphirkalt ◴[] No.45267528{3}[source]
Though you will see the code at least, when you are copy pasting it and if it is really only a few lines, you may be able to review it. Should review it of course.
replies(1): >>45269090 #
11. btown ◴[] No.45267701[source]
A brief but important point is that this primarily holds true in the context of rewriting/vendoring utilities yourself, not when discussing importing small vs. large dependencies.

Just because dependencies do a lot more than you need, doesn't mean you should automatically reach for the smallest dependency that fits your needs.

If you need 5 of the dozens of Lodash functions, for instance, it might be best to just install Lodash and let your build step shake out any unused code, rather than importing 5 new dependencies, each with far fewer eyes and release-management best practices than the Lodash maintainers have.

replies(3): >>45268248 #>>45268399 #>>45269728 #
12. philipwhiuk ◴[] No.45267705[source]
Do you have any evidence it wouldn't just make up code.
13. shakna ◴[] No.45268191[source]
Requiring you to audit both security and robustness on the LLM generated code.

Creating two problems, where there was one.

replies(2): >>45269859 #>>45271077 #
14. jay_kyburz ◴[] No.45268248{3}[source]
Yes, fewer, larger, trustworthy dependencies with tree shaking is the way to go if you ask me.
replies(1): >>45268390 #
15. kristianbrigman ◴[] No.45268298[source]
One interesting side effect of AI is that it makes it sometimes easy to just recreate the behavior, perhaps without even realizing it..
16. _puk ◴[] No.45268390{4}[source]
Almost like a standard library..
replies(2): >>45268741 #>>45272423 #
17. Terr_ ◴[] No.45268399{3}[source]
I think the level of protection you get from that depends on how the unused code detection interacts with whatever tricks someone is using for malicious code.
18. jay_kyburz ◴[] No.45268741{5}[source]
Yeah, but perhaps we could have different flavors. If you like functional style you could have a very functional standard library that doesn't mutate anything, or if you like object oriented stuff you could have classes of object with methods that mutate themselves. And the Typescript folks could have a strongly typed library.
19. const_cast ◴[] No.45269005[source]
This is already a thing, compiled languages have been doing this for decades. This is just C++ templates with extra steps.
20. LtWorf ◴[] No.45269090{4}[source]
If it's that little review the dependency.
replies(1): >>45272950 #
21. appreciatorBus ◴[] No.45269094{3}[source]
Sure but that's a one time vector. If the attacker didn't infiltrate the LLM before it generated the code, then the code is not going to suddenly go hostile like an npm package can.
22. reaperducer ◴[] No.45269164[source]
it's probably even less feasible to rewrite every single dependency from scratch.

When you code in a high-security environment, where bad code can cost the company millions of dollars in fines, somehow you find a way.

The sibling commenter is correct. You write what you can. You only import from trusted, vetted sources.

23. latexr ◴[] No.45269728{3}[source]
The argument wasn’t to import five dependencies, one for each of the functions, but to write the five functions yourself. Heck, you don’t even need to literally write them, check the Lodash source and copy them to your code.
replies(3): >>45269898 #>>45269917 #>>45272318 #
24. bennyg ◴[] No.45269859{3}[source]
I didn't say generate :) - in all seriousness, I think you could reasonably have it copy the code for e.g. lodash.merge() and paste it into your codebase without the headaches you're describing. IMO, this method would be practical for a majority of npm deps in prod code. There are some I'd want to rely on the lib (and its maintenance over time), but also... a sort function is a sort function.
replies(1): >>45270170 #
25. halflife ◴[] No.45269898{4}[source]
And then when node is updated and natively supports set intersections you would go back to your copied code and fix it?
replies(2): >>45270249 #>>45273825 #
26. mandevil ◴[] No.45269917{4}[source]
This might be fine for some utility functions which you can tell at a glance have no errors, but for anything complex, if you copy you don't get any of the bug/security fixes that upstream will provide automatically. Oh, now you need a shim of this call to work on the latest Chrome because they killed an api- you're on your own or you have to read all of the release notes for a dependency you don't even have! But taking a dependency on some other library is, as you note, always fraught. Especially because of transitive dependencies, you end up having quite a target surface area for every dep you take.

Whether to take a dependency is a tricky thing that really comes down to engineering judgement- the thing that you (the developer) are paid to make the calls on.

replies(1): >>45270873 #
27. shakna ◴[] No.45270170{4}[source]
LLMs don't copy and paste. They ingest and generate. The output will always be a generated something.
replies(2): >>45270709 #>>45281752 #
28. 8note ◴[] No.45270175[source]
is it that infeasible with LLMs?

a lor of these dependencies are higher order function definitions, which never change, and could be copy/pasted around just fine. they're never gonna change

29. skydhash ◴[] No.45270249{5}[source]
If it works, why do so? Unless there's a clear performance boost, and if so you already know the code and can quickly locate your interpreted version.

Or At the time of adding you can add a NOTE or FIXME comment stating where you copied it from. A quick grep for such keyword can give you a nice overview of nice to have stuff. You can also add a ticket with all the details if you're using a project management tool and resuscitate it when that hypothetical moment happens.

30. motorest ◴[] No.45270363[source]
> If it's not feasible to audit every single dependency, it's probably even less feasible to rewrite every single dependency from scratch.

There is no need to rewrite dependencies. Sometimes it just so happens that a project can live without outputting fancy colorful text to stdout, or doesn't need to spread transitive dependencies on debug utilities. Perhaps these concerns should be a part of the standard library, perhaps these concerns are useless.

And don't get me started on bullshit polyfill packages. That's an attack vector waiting to be exploited.

31. smrtinsert ◴[] No.45270519[source]
Its much more feasible these days. These days for my personal projects I just have CC create only a plain html file with raw JS and script links.
32. autoexec ◴[] No.45270529[source]
> It isn't feasible to audit every line of every dependency, just as it's not possible to audit the full behavior of every employee that works at your company.

Your employees are carefully vetted before hiring. You've got their names, addresses, and social security numbers. There's someone you're able to hold accountable if they steal from you or start breaking everything in the office.

This seems more like having several random contractors who you've never met coming into your business in the middle of night. Contractors that were hired by multiple anonymous agencies you just found online somewhere with company names like gkz00d or 420_C0der69 who you've also never even spoken to and who have made it clear that they can't be held accountable for anything bad that happens. Agencies that routinely swap workers into or out of various roles at your company without asking or telling you, so you don't have any idea who the person working in the office is, what they're doing, or even if they're supposed to be there.

"To make thing easier for us we want your stuff to require the use of a bunch of code (much of which does things you don't even need) that we haven't bothered looking at because that'd be too much work for us. Oh, and third parties we have no relationship with control a whole bunch of that code which means it can be changed at any moment introducing bugs and security issues we might not hear about for months/years" seems like it should be a hard sell to a boss or a client, but it's sadly the norm.

Assuming that something is going to go wrong and trying to limit the inevitable damage is smart, but limiting the amount of untrustworthy code maintained by the whims of random strangers is even better. Especially when the reasons for including something that carries so much risk is to add something trivial or something you could have just written yourself in the first place.

replies(2): >>45272298 #>>45273210 #
33. TheBicPen ◴[] No.45270709{5}[source]
In 2022, sure. But not today. Even something as simple as generating and running a `git clone && cp xyz` command will create code not directly generated by the LLM.
replies(1): >>45288106 #
34. vasco ◴[] No.45270855{3}[source]
What paper?
replies(1): >>45271127 #
35. jonquest ◴[] No.45270873{5}[source]
The massive amount of transitive dependencies is exactly the problem with regard to auditing them. There are successful businesses built solely around auditing project dependencies and alerting teams of security issues, and they make money at all because of the labor required to maintain this machine.

It’s not even a judgement call at this point. It’s more aligned with buckling your seatbelt, pointing your car off the road, closing your eyes, flooring it and hoping for a happy ending.

36. hshdhdhj4444 ◴[] No.45271035[source]
I agree with this but the problem is that a lot of the extra stuff dependencies do is indeed to protect from security issues.

If you’re gonna reimplement only thr code you need from a dependency, it’s hard to know of the stuff you’re leaving out how much is just extra stuff you don’t need and how much might be security fixes that may not be apparent to you but the dependency by virtue of being worked upon and used by many people has fixed.

37. vFunct ◴[] No.45271065[source]
I'm using LLMs to write stuff that would normally be in dependencies, mostly because I don't want to learn how to use the dependency, and writing a new one from scratch is really easy with LLMs.
replies(1): >>45272434 #
38. vFunct ◴[] No.45271077{3}[source]
LLMs can do the audits now.
39. what ◴[] No.45271127{4}[source]
Presumably this: https://github.com/zahlman/paper
replies(1): >>45278782 #
40. lukan ◴[] No.45272158{3}[source]
I did not say to do blind copy paste.

A few lines of code can be audited.

41. skwashd ◴[] No.45272298{3}[source]
> This seems more like having several random contractors who you've never met coming into your business in the middle of night. [...] Agencies that routinely swap workers into or out of various roles at your company without asking or telling you, so you don't have any idea who the person working in the office is, what they're doing, or even if they're supposed to be there.

Sounds very similar to how global SIs staff enterprise IT contracts.

42. cluckindan ◴[] No.45272318{4}[source]
You have obviously never checked the Lodash source.
replies(1): >>45273560 #
43. baq ◴[] No.45272423{5}[source]
I wanted to make a joke about

   npm install stdlib 
…but double checked before and @stdlib/stdlib has 58 dependencies, so the joke preempted me.
44. baq ◴[] No.45272434{3}[source]
Age of bespoke software is here. Did you have any hard to spot non-obvious bugs in these code units?
45. lukan ◴[] No.45272950{5}[source]
The difference is, the dependency can change and is usually way harder to audit. Subfolders in subfolder, 2 lines here in a file, 3 line there vs locking at some files and check what they do.
46. xorcist ◴[] No.45273210{3}[source]
That hit much too close to reality. It's exactly like that. Even the names were spot on!
47. latexr ◴[] No.45273560{5}[source]
The point here isn’t a specific library. It’s not even one specific language or runtime. No one is talking about literally five functions. Let’s not be pedantic and lose sight of the major point.
replies(1): >>45273568 #
48. cluckindan ◴[] No.45273568{6}[source]
I get that, but if you’ve ever tried to extract a single utility function from lodash, you know that it may not be as simple as copy-pasting a single function.
replies(1): >>45274322 #
49. ClikeX ◴[] No.45273825{5}[source]
If you won't, do you expect the maintainer of some micro package to do that?
50. zelphirkalt ◴[] No.45274322{7}[source]
If you are going to be that specific, then it would be good to post an example. If I remember correctly, lodash has some functions, that would be table stakes in functional languages, or easily built in functional languages. If such a function is difficult to extract, then it might be a good candidate to write in JS itself, which does have some of the typical tools, like map, reduce, and things like compose are easy to write oneself and part of every FP beginner tutorial. If such a function is difficult to extract, then perhaps lodash's design is not all that great. Maybe one could also copy them from elsewhere, where the code is more modular.

But again, if the discussion is going to be that specific, then you would need to provide actual examples, so that we could judge, whether we would implement that ourselves or it would be difficult to do so. Note, that often it is also not required for ones use-case, to have a 100% matching behavior either. The goal is not to duplicate lodash. The purpose of the extracted or reimplemented function would still be ones own project, where the job of that function might be much more limited.

replies(1): >>45275698 #
51. cluckindan ◴[] No.45275698{8}[source]
Let’s start with something simple, like difference().

https://github.com/lodash/lodash/blob/main/dist/lodash.js#L7...

So you also need to copy isArrayLikeObject, baseDifference and baseFlatten.

For baseDifference, you also need to copy arrayMap and baseUnary.

For baseFlatten, you also need to copy arrayPush.

For isArrayLikeObject, you also need to copy isArrayLike and isObjectLike.

For isArrayLike, you also need to copy isLength and isFunction.

For isFunction, you also need to copy isObject and baseGetTag.

For baseGetTag, you also need to copy getRawTag and objectToString.

I don’t have time to dig any deeper, just use tree-shaking ffs.

replies(1): >>45276479 #
52. zelphirkalt ◴[] No.45276479{9}[source]
OK in this case it looks like it is doing a lot of at runtime checking of arguments to treat them differently, based on what type of argument they are. If we restrict use to only work with arrays, or whatever we have in our project, where we need `difference`, then it should become much simpler and an easy rewrite. An alternative could be to have another argument, that is the function that gives us the `next` thing. Then the logic for that is to be specified by the caller.

Tree shaking however, will not help you, if you have to first install a library using NPM. It will only help you reduce overhead in the code served to a browser. Malicious code can run much earlier, and would be avoided, if you rewrite or extract relevant code from a library, avoiding to install the library using NPM. Or is there some pre-installation tree shaking, that I am unaware of? That would actually be interesting.

replies(1): >>45277142 #
53. cluckindan ◴[] No.45277142{10}[source]
I guess that pre-installation tree shaking in this case is installing ’lodash.difference’ instead of ’lodash’. :)
54. zahlman ◴[] No.45278782{5}[source]
Yes, that. I didn't want to be too spammy, especially since I honestly haven't been getting much of anything done recently (personal reasons).
55. lgas ◴[] No.45281752{5}[source]
You can give an LLM access to tools that it can invoke to actually copy and paste.
56. boomlinde ◴[] No.45288106{6}[source]
In what way do you think this rebuts the message you responded to?