←back to thread

466 points 0x63_Problems | 5 comments | | HN request time: 0.623s | source
Show context
perrygeo ◴[] No.42138092[source]
> Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever.

This mirrors my experience using LLMs on personal projects. They can provide good advice only to the extent that your project stays within the bounds of well-known patterns. As soon as your codebase gets a little bit "weird" (ie trying to do anything novel and interesting), the model chokes, starts hallucinating, and makes your job considerably harder.

Put another way, LLMs make the easy stuff easier, but royally screws up the hard stuff. The gap does appear to be widening, not shrinking. They work best where we need them the least.

replies(24): >>42138267 #>>42138350 #>>42138403 #>>42138537 #>>42138558 #>>42138582 #>>42138674 #>>42138683 #>>42138690 #>>42138884 #>>42139109 #>>42139189 #>>42140096 #>>42140476 #>>42140626 #>>42140809 #>>42140878 #>>42141658 #>>42141716 #>>42142239 #>>42142373 #>>42143688 #>>42143791 #>>42151146 #
1. glouwbug ◴[] No.42140626[source]
Ironically enough I’ve always found LLMs work best when I don’t know what I’m doing
replies(2): >>42141007 #>>42141805 #
2. mindok ◴[] No.42141007[source]
Is that because you can’t judge the quality of the output?
replies(1): >>42150194 #
3. hambandit ◴[] No.42141805[source]
I find this perspective both scary and exciting. I'm curious, how do you validate the LLM's output? If you have a way to do this, and it's working. Then that's amazing. If you don't, how are you gauging "work best"?
replies(1): >>42150241 #
4. glouwbug ◴[] No.42150194[source]
Exactly
5. glouwbug ◴[] No.42150241[source]
I gauge what work's best if I can already do what I am asking it to do, and that comes from years of studying and trial and error experience without LLMs. I have no way of verifying what's a hallucination unless I am an expert