←back to thread

Ask HN: Any insider takes on Yann LeCun's push against current architectures?

385 points vessenes | 1 comments | 10 Mar 25 19:41 UTC | HN request time: 0.266s | source

So, Lecun has been quite public saying that he believes LLMs will never fix hallucinations because, essentially, the token choice method at each step leads to runaway errors -- these can't be damped mathematically.

In exchange, he offers the idea that we should have something that is an 'energy minimization' architecture; as I understand it, this would have a concept of the 'energy' of an entire response, and training would try and minimize that.

Which is to say, I don't fully understand this. That said, I'm curious to hear what ML researchers think about Lecun's take, and if there's any engineering done around it. I can't find much after the release of ijepa from his group.

Show context
TrainedMonkey ◴[14 Mar 25 18:03 UTC] No.43365343[source]▶
This is a somewhat nihilistic take with an optimistic ending. I believe humans will never fix hallucinations. Amount of totally or partially untrue statements people make is significant. Especially in tech, it's rare for people to admit that they do not know something. And yet, despite all of that the progress keeps marching forward and maybe even accelerating.
replies(5): >>43365376 #>>43366746 #>>43367326 #>>43368390 #>>43368923 #
ketzo ◴[14 Mar 25 18:06 UTC] No.43365376[source]▶
Yeah, I think a lot of people talk about "fixing hallucinations" as the end goal, rather than "LLMs providing value", which misses the forest for the trees; it's obviously already true that we don't need totally hallucination-free output to get value from these models.
replies(1): >>43368890 #
1. mdp2021 ◴[15 Mar 25 00:45 UTC] No.43368890[source]▶
Even as language models can partially solve a few problems, we remain with the problem of achieving Artificial General Intelligence, that the presence of LLMs has exacerbated because they so often reveal to be artificial morons.

Intelligence finds solutions - actual, solid solutions.

More than "fixing" hallucinations, the problem is going beyond them (arriving to "sobriety").