←back to thread

124 points alphadelphi | 6 comments | | HN request time: 0.433s | source | bottom
Show context
antirez ◴[] No.43594641[source]
As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between the words of a text and the gist of the content, still he was saying that at the time of GPT3.5 I believe. Then he adjusted this vision when talking with Hinton publicly, saying "I don't deny there is more than just probabilistic thing...". He started saying: not longer just simply probabilistic but they can only regurgitate things they saw in the training set, often explicitly telling people that novel questions could NEVER solved by LLMs, with examples of prompts failing at the time he was saying that and so forth. Now reasoning models can solve problems they never saw, and o3 did huge progresses on ARC, so he adjusted again: for AGI we will need more. And so forth.

So at this point it does not matter what you believe about LLMs: in general, to trust LeCun words is not a good idea. Add to this that LeCun is directing an AI lab that as the same point has the following huge issues:

1. Weakest ever LLM among the big labs with similar resources (and smaller resources: DeepSeek).

2. They say they are focusing on open source models, but the license is among the less open than the available open weight models.

3. LLMs and in general all the new AI wave puts CNNs, a field where LeCun worked (but that didn't started himself) a lot more in perspective, and now it's just a chapter in a book that is composed mostly of other techniques.

Btw, other researchers that were in the LeCun side, changed side recently, saying that now "is different" because of CoT, that is the symbolic reasoning they were blabling before. But CoT is stil regressive next token without any architectural change, so, no, they were wrong, too.

replies(15): >>43594669 #>>43594733 #>>43594747 #>>43594812 #>>43594852 #>>43595292 #>>43595501 #>>43595519 #>>43595562 #>>43595668 #>>43596291 #>>43596309 #>>43597354 #>>43597435 #>>43614487 #
gcr ◴[] No.43594669[source]
Why is changing one’s mind when confronted with new evidence a negative signifier of reputation for you?
replies(6): >>43594696 #>>43594815 #>>43594919 #>>43595008 #>>43595180 #>>43595298 #
danielmarkbruce ◴[] No.43595008[source]
If you need basically rock solid evidence of X before you stop saying "this thing cannot do X", then you shouldn't be running a forward looking lab. There are only so many directions you can take, only so many resources at your disposal. Your intuition has to be really freakishly good to be running such a lab.

He's done a lot of amazing work, but his stance on LLMs seems continuously off the mark.

replies(2): >>43595040 #>>43596502 #
SJC_Hacker ◴[] No.43595040[source]
The list of great minds who thought that "new fangled thing is nonsense" and later turned out to be horribly wrong is quite long and distinguished
replies(3): >>43595318 #>>43595476 #>>43595700 #
1. fragmede ◴[] No.43595318[source]
> Heavier-than-air flying machines are impossible.

-Lord Kelvin. 1895

> I think there is a world market for maybe five computers. Thomas Watson, IBM. 1943

> On talking films: “They’ll never last.” -Charlie Chaplin.

> This ‘telephone’ has too many shortcomings… -William Orton, Western Union. 1876

> Television won’t be able to hold any market -Darryl Zanuck, 20th Century Fox. 1946

> Louis Pasteur’s theory of germs is ridiculous fiction. -Pierre Pachet, French physiologist.

> Airplanes are interesting toys but of no military value. — Marshal Ferdinand Foch 1911

> There’s no chance the iPhone is going to get any significant market share. — Steve Ballmer, CEO Microsoft CEO. 2007

> Stocks have reached a permanently high plateau. — Irving Fisher, Economist. 1929

> Who the hell wants to hear actors talk? —Harry Warner, Warner Bros. 1927

> By 2005, it will become clear that the Internet’s impact on the economy has been no greater than the fax machine. -Paul Krugman, Economist. 1998

replies(4): >>43595556 #>>43595725 #>>43595745 #>>43596122 #
2. ksec ◴[] No.43595556[source]
I am just wondering did you have this all somehow saved up or did you pull it out of somewhere? Amazing list of things. Thank You.
replies(1): >>43595774 #
3. MrMcCall ◴[] No.43595725[source]
I'm pretty sure that Lord Kelvin was also in the cohort of fools that bullied Boltzmann to his suicide.
4. mdp2021 ◴[] No.43595745[source]
An important number of those remarks were based on a snapshot of the state of the technology: a fault in not seeing the potential evolution.

Examples of people who could not see non (in some way) dead-ends do not cancel examples of people who correctly saw dead-ends. The lists may even overlap ("if it remains that way it's a dead-end").

5. fragmede ◴[] No.43595774[source]
Gosh no. I knew most of that list but I'll be honest and tell you that I used ChatGPT to come up with it. it's a collection of quotes to begin with so I think that's okay. I'm not passing off someone else's writing as my own, I'm explicitly quoting them.
6. jcalvinowens ◴[] No.43596122[source]
In fairness to Irving Fisher: if you bought into the market at its peak in 1929, you wouldn't recover your original investment until about 1960.