←back to thread

Ask HN: Any insider takes on Yann LeCun's push against current architectures?

385 points vessenes | 2 comments | 10 Mar 25 19:41 UTC | HN request time: 0s | source

So, Lecun has been quite public saying that he believes LLMs will never fix hallucinations because, essentially, the token choice method at each step leads to runaway errors -- these can't be damped mathematically.

In exchange, he offers the idea that we should have something that is an 'energy minimization' architecture; as I understand it, this would have a concept of the 'energy' of an entire response, and training would try and minimize that.

Which is to say, I don't fully understand this. That said, I'm curious to hear what ML researchers think about Lecun's take, and if there's any engineering done around it. I can't find much after the release of ijepa from his group.

1. in3d ◴[15 Mar 25 02:23 UTC] No.43369367[source]▶
I don't think it's a coincidence that he is interested in non-LLM solutions, since he mentioned last year on Twitter that he doesn't have an internal monologue (I hope this is not taken as disparaging of him in any way). His criticisms of LLMs never made sense, and the success of reasoning models has shown him to be definitely wrong.
replies(1): >>43369712 #
2. WiSaGaN ◴[15 Mar 25 03:23 UTC] No.43369712[source]▶
Yes, it is fascinating that humans can have such seemingly fundamental differences in how they function 'under the hood.' I also have a friend who is highly intelligent—they earned a STEM PhD from one of the best universities in the world—yet they struggle to follow complex movie plots, despite having a photographic memory. It would be interesting to develop mirror LLMs (or Large Anything Models) for all these different types of brains so we can study how exactly these traits manifest and interact.