←back to thread

358 points andrewstetsenko | 2 comments | | HN request time: 0.759s | source
Show context
voidhorse ◴[] No.44361339[source]
Hopefully a CEO finally tempering some expectations and the recent Apple paper bring some sanity back into the discourse around these tools.[^1]

Are they cool and useful? Yes.

Do they reason? No. (Before you complain, please first define reason).

Are they end all be all of all problem solving and the dawn of AGI? Also no.

Once we actually bring some rationality back into the radius of discourse maybe we'll actually begin to start figuring out how these things actually fit into an engineering workflow and stop throwing around ridiculous terms like vibe coding.

If you are an engineer, you are signing up to build rigorously verified and validated system, preferably with some amount of certainty about their behavior under boundary conditions. All the current hype-addled discussion around LLM seems to have had everything but correctness as it's focus.

[^1]: It shouldn't take a CEO but many people, even technologists, who should be more rational about whose opinions they deem worthy of consideration, m seem to overvalue the opinions of the csuite for some bizarre, inexplicable reason.

replies(1): >>44362883 #
1. crackalamoo ◴[] No.44362883[source]
The recent Apple paper seemed pretty flawed. Anthropic/Open Philanthropy did a rebuttal paper, The Illusion of the Illusion of Thinking.

> Do they reason? No. (Before you complain, please first define reason).

Defining reasoning is the problem. No, they don't reason in the same way as humans. But they seem to be able to go through reasoning steps in some important way. It's like asking if submarines swim.

replies(1): >>44366859 #
2. voidhorse ◴[] No.44366859[source]
I completely agree. The difference is, the utility of the submarine and its sales strategy did not depend on convincing human beings that it does swim like them.

The opposite is true of LLMs. Much of the sales pitch of these companies is that these things are capable of some form of reasoning that is in 1:1 correspondence with human reason. Not true. That's what makes this whole segment of the industry snake oil. It's not complete bunk because they do have some utility, but these companies know what they are doing when they use the term reason in all their marketing and papers. They are, in fact, trying to sell you the submarine because it can "swim".