←back to thread

Ask HN: Any insider takes on Yann LeCun's push against current architectures?

385 points vessenes | 2 comments | 10 Mar 25 19:41 UTC | HN request time: 0s | source

So, Lecun has been quite public saying that he believes LLMs will never fix hallucinations because, essentially, the token choice method at each step leads to runaway errors -- these can't be damped mathematically.

In exchange, he offers the idea that we should have something that is an 'energy minimization' architecture; as I understand it, this would have a concept of the 'energy' of an entire response, and training would try and minimize that.

Which is to say, I don't fully understand this. That said, I'm curious to hear what ML researchers think about Lecun's take, and if there's any engineering done around it. I can't find much after the release of ijepa from his group.

1. ilaksh ◴[14 Mar 25 22:02 UTC] No.43367746[source]▶
I don't think you need to be an ML researcher to understand his point of view. He wants to do fundamental research. Optimizing LLMs is not fundamental research. There are numerous other potential approaches, and it's obvious that LLMs have weaknesses that other approaches could tackle.

If he was Hinton's age then maybe he would also want to retire and be happy with transformers and LLMs. He is still an ambitious researcher that wants to do foundational research to get to the next paradigm.

Having said all of that, it is a misjudgement for him to be disparaging the incredible capabilities of LLMs to the degree he has.

replies(1): >>43367769 #
2. moron4hire ◴[14 Mar 25 22:05 UTC] No.43367769[source]▶
> it is a misjudgement for him to be disparaging the incredible capabilities of LLMs to the degree he has.

Jeez, you'd think he kicked your dog.