It's so depressing to watch so many smart people spend their considerable talents on the generation of utter garbage and the erosion of the social fabric of society.
Not just by companies. We see this from enthusiastic consumers as well, on this very forum. Or it might just be astroturfing, it's hard to tell.
The mantra is that in order to extract value from LLMs, the user must have a certain level of knowledge and skill of how to use them. "Prompt engineering", now reframed as "context engineering", has become this practice that separates anyone who feels these tools are wasting their time more than they're helping, and those who feel that it's making them many times more productive. The tools themselves are never the issue. Clearly it's the user who lacks skill.
This narrative permeates blog posts and discussion forums. It was recently reinforced by a misinterpretation of a METR study.
To be clear: using any tool to its full potential does require a certain skill level. What I'm objecting to is the blanket statement that people who don't find LLMs to be a net benefit to their workflow lack the skills to do so. This is insulting to smart and capable engineers with many years of experience working with software. LLMs are not this alien technology that require a degree to use correctly. Understanding how they work, feeding them the right context, and being familiar with the related tools and concepts, does not require an engineering specialization. Anyone claiming it does is trying to sell you something; either LLMs themselves, or the idea that they're more capable than those criticizing this technology.
"This LLM is able to capably output useful snippets of code for Python. That's useful."
and
"I tried to get an LLM to perform a niche task with a niche language, it performed terribly."
I think the right synthesis is that there are some tasks the LLMs are useful at, some which they're not useful at; practically, it's useful to be able to know what they're useful for.
Or, if we trust that LLMs are useful for all tasks, then it's practically useful to know what they're not good at.
As was discussed on a subthread on HN a few weeks ago, the key to developing successful LLM applications is going to be figuring out how to put in the necessary business-specific guardrails with a fallback to a human-in-the-loop.
The difference is that humans eventually learn. We accept that someone who joins a team will be net-negative for the first few days, weeks, or even months. If they keep making the same mistakes that were picked out in their first code review, as LLMs do, eventually we fire them.
Worst case, if they don't do this automatically, you can simply "teach" them by updating the prompt to watch for a specific mistake (similar to how we often add a test when we catch a bug.)
But it need not even be that cumbersome. Even weaker models do surprisingly well with broad guidelines. Case in point: https://news.ycombinator.com/item?id=42150769
The thing is that there's no way to objectively measure this. Benchmarks are often gamed, and like a sibling comment mentioned, the output is not stable.
Also, everyone has different criteria for what constitutes "good". To someone with little to no programming experience, LLMs would feel downright magical. Experienced programmers, or any domain expert for that matter, would be able to gauge the output quality much more accurately. Even among the experienced group, there are different levels of quality criteria. Some might be fine with overlooking certain issues, or not bother checking the output at all, while others have much higher standards of quality.
The problem is when any issues that are pointed out are blamed on the user, instead of the tool. Or even worse: when the issues are acknowledged, but are excused as "this is the way these tools work."[1,2]. It's blatant gaslighting that AI companies love to promote for obvious reasons.
Sure. But isn't that a bit like if someone likes VSCode, & someone likes Emacs.. the first method of comparison I'm reaching for isn't "what objective metrics do you have", so much as "how do you use it?".
> > This is insulting to smart and capable engineers with many years of experience working with software.
> Experienced programmers, or any domain expert for that matter, would be able to gauge the output quality much more accurately.
My experience is that smart and capable engineers have varying opinions on things. -- "What their opinion is" is less interesting than "why they have the opinion".
I would be surprised, though, if someone were to boast about their experience/skills, & claim they were unable to find any way to use LLMs effectively.
And even if it did, a certain degree of non-determinism is actually desirable. The most probable tokens might not be correct, and randomness is partly responsible for what humans interpret as "creativity". Even hallucinations are desirable in some applications (art, entertainment, etc.).
[1]: https://medium.com/google-cloud/is-a-zero-temperature-determ...
Put an LLM on documentation or man pages. Tell the LLM to output a range of lines, and the system actually looks up those lines and quotes them. The overall effect is that the LLM can do some free-form output, but is expected to provide a citation to support its claims; and the citation can't be hallucinated, since the LLM doesn't generate the citation, a plain old computer program does.
And we haven't seen LLMs integrated with type systems yet. There are very powerful type systems, like dependent types, that can prove things like "this function returns a list of sorted number", and the type system ensures that is ALWAYS true [0], at compile time. You have to write a lot of proof code to help the compiler do these checks at compile time, but if a LLM can write those proofs, we can trust they are correct, because only correct proofs will compile.
[0]: Or rather, almost always true. There's always the possibility of running out of memory or the power goes out.
I've used a tool to do a task today. I used a suction sandblasting machine to remove corrosion from a part.
Without the tool, had I wanted to remove the corrosion, I would've spent all day (if not more) scraping it with sandpaper (is that a tool too? With the skin of my hands then?) - this would have been tedious and could have taken me all day, scraping away millimeter by millimeter.
With the machine, it took me about 3 minutes. I necessitated 4-5 minutes of training to attain this level of expertise.
The worth of this machine is undeniable.
How is it that LLMs are not at all so undeniably efficient? I keep hearing people tell me how they will take everyones job, but it seems like the first faceplant from all the big tech companies.
(Maybe second after Meta's VR stuff)
Formulaic, unspecific in results while making extraordinary claims, and always of a specific upbeat tenor.
For example, people try to compare this LLM tech with the automation of the car manufacturing industry. That analogy is a terrible one, because machines build better cars and are much more reliable than humans.
LLMs don't build better software, they build bad software faster.
Also, as a tool, LLMs discourage understanding in a way that no other tool does.
LLMs are literally performing useful functions today and they're not going away. Are they AGI? No, but so what?
There is waaay too much projecting and philosophizing going on in these comments and not enough engineering-minded comments from objective observers.
Is AI hyped? Sure. Are LLMs overshadowing other approaches? Sure. Are LLMs inefficient? Somewhat. Do they have problems like hallucinations? Yes. Do they produce useful output? Yes.