Most active commenters
  • haswell(3)
  • war-is-peace(3)
  • philjohn(3)
  • ninetyninenine(3)

←back to thread

Why AI is still dumb and not scary at all (pt. 1)

(tejo.substack.com)
63 points tejonutella | 12 comments | 08 Mar 25 21:25 UTC | HN request time: 1.848s | source | bottom
Show context
haswell ◴[08 Mar 25 22:20 UTC] No.43304096[source]▶
This is a frustrating piece on many levels, but mostly because it doesn't really scratch the surface of what is worrisome about AI. It sets up straw men and knocks them down, but not much else.

It seems to boil down to:

1. LLMs aren't actually "learning" the way humans do so we shouldn't be worried

2. LLMs don't actually "understand" anything so we shouldn't be worried

3. Technology has always been advancing and we've always been freaking out about that, so we shouldn't be worried

4. If your job is automate-able, it probably should be eliminated anyway

What's scary is not that these models are smarter than us, but that we are dumb enough to deploy them in critical contexts and trust the output they generate.

What's scary isn't that these models are so good they'll replace us, but that despite how limited they are, someone will make the decision to replace humans anyway.

What's scary isn't that LLMs will displace good developers, but that LLMs put the power of development in the hands of people who have no idea what they're wielding.

> Sure, with a millions upon millions of training examples, of course you can mimic intelligence. If you already know what’s going to be on the test, common patterns for answers in the test, or even the answer key itself, then are you really intelligent? OR are you just regurgitating information from billions of past tests?

How different are humans from this description in actuality? What are we if not the results of a process that has been optimized by millions upon millions of iterations over long periods of time?

replies(2): >>43304181 #>>43304293 #
war-is-peace ◴[08 Mar 25 22:31 UTC] No.43304181[source]▶
> What's scary isn't that these models are so good they'll replace us, but that despite how limited they are, someone will make the decision to replace humans anyway.

is this a real threat? if a system/company decides to replace a human with something that is less capable wouldn't that just result in it becoming irrelevant/bankrupt as it is replaced by other companies doing the same thing the more efficient (and in this case traditional) way?

replies(3): >>43304202 #>>43304234 #>>43304323 #
1. ed-209 ◴[08 Mar 25 22:35 UTC] No.43304202[source]▶
Not necessarily. Imagine a health insurance provider even partially automating their claim (dis)approval process - it could be both lucrative and devastating.
replies(3): >>43304246 #>>43304267 #>>43304321 #
2. haswell ◴[08 Mar 25 22:43 UTC] No.43304246[source]▶
Adding to this, government use cases would be most likely to cause issues because they’re often relevant regardless of how badly they suck.

There are already active discussions about AI being used in government for “efficiency” reasons.

replies(2): >>43304294 #>>43304906 #
3. war-is-peace ◴[08 Mar 25 22:45 UTC] No.43304267[source]▶
worse for the consumer or the provider? if the llm is going to fundamentally do a "worse" job no matter what the incentive (whatever that is, maximising profit, maximising claims, whatever that may be) we will end up with the "more efficient" system in charge

the counterpoint to this (which i guess is the tenet of the original comment?) is that hype can shadow good judgement for a short period of time?

4. war-is-peace ◴[08 Mar 25 22:49 UTC] No.43304294[source]▶
i suppose that links back to the other comment i made - is hype the root issue you are trying to get at?

would be interesting to see what examples there are of this in recent history

replies(1): >>43304570 #
5. philjohn ◴[08 Mar 25 22:52 UTC] No.43304321[source]▶
Already happened. With predictable results.

https://www.youtube.com/shorts/Y7QPXzDmloI?embeds_referring_...

replies(1): >>43304579 #
6. haswell ◴[08 Mar 25 23:30 UTC] No.43304570{3}[source]▶
I'm not entirely sure what you're getting at re: hype.

While there is undoubtedly a lot of hype around these tools right now, that hype is based on a pretty major leap in technology that has fundamentally altered the landscape going forward. There are some great use cases that legitimize some of the hype.

As far as concrete examples, see the sibling comment with the anecdote regarding health insurance denial. There are also portions of the tech industry focused on rolling these tools out in business environments. They're publicly reporting their earnings, and discussing the role AI is playing in major business deals.

Look at players like Salesforce, ServiceNow, Atlassian, etc. They're all rapidly rolling out various AI capabilities into their existing customer bases. They have giant sales forces all actively pushing these capabilities. They also sell to governments. Hype or not, it adds up to real world outcomes.

Public statements by Musk about his intention to use AI also come to mind, and he's repeatedly shown a willingness to break things in the pursuit of his goals.

7. ninetyninenine ◴[08 Mar 25 23:30 UTC] No.43304579[source]▶
is that even legal?
replies(1): >>43304588 #
8. philjohn ◴[08 Mar 25 23:31 UTC] No.43304588{3}[source]▶
Who's going to stop them?
replies(1): >>43310859 #
9. trescenzi ◴[09 Mar 25 00:23 UTC] No.43304906[source]▶
A similar issue arises with health insurance. Using AI to evaluate claims is a huge efficiency play and you don’t have much ability to fight it if something goes wrong. And even if you can these decisions can be life or death in the short term and human intervention usually takes time.
10. ninetyninenine ◴[09 Mar 25 16:42 UTC] No.43310859{4}[source]▶
Lawyers
replies(1): >>43313375 #
11. philjohn ◴[09 Mar 25 20:25 UTC] No.43313375{5}[source]▶
The current admin is unlikely to get the DOJ to do anything, so that leaves you trying to sue them yourselves, and they can afford more expensive lawyers and to gum up the proceedings until you go bankrupt or give up.

If you haven't already signed away your right to sue with an arbitration clause.

replies(1): >>43313505 #
12. ninetyninenine ◴[09 Mar 25 20:38 UTC] No.43313505{6}[source]▶
class action. Someone sues on behalf of all customers of the insurance company saying the insurance policy is fraudulent. The company is executing policies based on ai or rolling the dice. The insurance company owes all customers balance paid on policy and all costs associated with attempting to use fraudulent policy.