←back to thread

The wall confronting large language models

(arxiv.org)
170 points PaulHoule | 3 comments | 03 Sep 25 11:40 UTC | HN request time: 0.647s | source
Show context
Scene_Cast2 ◴[03 Sep 25 17:58 UTC] No.45118686[source]▶
The paper is hard to read. There is no concrete worked-through example, the prose is over the top, and the equations don't really help. I can't make head or tail of this paper.
replies(3): >>45118775 #>>45119154 #>>45120083 #
lumost ◴[03 Sep 25 18:06 UTC] No.45118775[source]▶
This appears to be a position paper written by authors outside of their core field. The presentation of "the wall" is only through analogy to derivatives on the discrete values computer's operate in.
replies(2): >>45119119 #>>45119709 #
jibal ◴[03 Sep 25 19:45 UTC] No.45119709[source]▶
If you look at their other papers, you will see that this is very much within their core field.
replies(3): >>45119914 #>>45120336 #>>45124453 #
JohnKemeny ◴[03 Sep 25 20:57 UTC] No.45120336[source]▶
He's a chemist. Lots of chemists and physicists like to talk about computation without having any background in it.

I'm not saying anything about the content, merely making a remark.

replies(3): >>45120611 #>>45122263 #>>45122690 #
1. chermi ◴[03 Sep 25 21:29 UTC] No.45120611[source]▶
You're really not saying anything? Just a random remark with no bearing?

Seth Lloyd, Wolpert, Landauer, Bennet, Fredkin, Feynman, Sejnowski, Hopfield, Zechinna, parisi,mezard, and zdebvora, Crutchfeld, Preskill, Deutsch, Manin, Szilard, MacKay....

I wish someone told them to shut up about computing. And I wouldn't dare claim von Neumann as merely a physicist, but that's where he was coming from. Oh and as much as I dislike him, Wolfram.

replies(1): >>45124354 #
2. JohnKemeny ◴[04 Sep 25 06:56 UTC] No.45124354[source]▶
As you note, some physicists do have computing backgrounds. I'm not suggesting they can't do computer science.

But today, most people hold opinions about LLMs, both as to their limits and their potential, without any real knowledge of computational linguistics nor of deep learning.

replies(1): >>45140626 #
3. chermi ◴[05 Sep 25 16:43 UTC] No.45140626[source]▶
Huh? Have you heard of learning something new? Physicists and scientists at large are pretty good at it. Do you want some certification program to determine who's allowed to opine? If someone is wrong, tell them and show them they're wrong. Don't preemptively dismiss ideas based on some authority mechanism.

Here's another example in case you still don't get the point - Schrodinger had no business talking about biology because he wasn't trained in it, right? Nevermind him being ahead of the entire field on understanding the role of "DNA"(yet undiscovered, but he correctly posited the crystal-ish structure) and information in evolution and inspiring Watson's quest to figure out DNA.

Judge ideas on the merit of the idea itself. It's not about whether they have computing backgrounds, its about the ideas.

Hell, look at the history of deep learning with Minsky's book. Sure glad everyone listened to the linguistics expert there...