Most active commenters
  • nomel(5)
  • JimDabell(5)
  • gortok(3)
  • tucnak(3)
  • (3)

←back to thread

1257 points adrianh | 35 comments | | HN request time: 1.815s | source | bottom
1. gortok ◴[] No.44495659[source]
I think folks have taken the wrong lesson from this.

It’s not that they added a new feature because there was demand.

They added a new feature because technology hallucinated a feature that didn’t exist.

The savior of tech, generative AI, was telling folks a feature existed that didn’t exist.

That’s what the headline is, and in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again, because next time it might not be so benign as it was this time.

replies(7): >>44495919 #>>44496083 #>>44496091 #>>44497641 #>>44498195 #>>44500852 #>>44505736 #
2. lexandstuff ◴[] No.44495919[source]
Sometimes you just have to deal with the world as it is, not how you think it should be.
replies(1): >>44496062 #
3. gortok ◴[] No.44496062[source]
Is it your argument that the folks that make generative AI applications have nothing to improve from this example?
replies(1): >>44502343 #
4. nomel ◴[] No.44496083[source]
> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again

This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its current imperfect state.

The current state of LLM is useful for what it's useful for, warnings of hallucinations are present on every official public interface, and its limitations are quickly understood with any real use.

Nearly everyone in AI research is working on this problem, directly or indirectly.

replies(3): >>44496098 #>>44496511 #>>44496702 #
5. gortok ◴[] No.44496098[source]
No one is “requiring perfection”, but hallucination is a major issue and is in the opposite direction of the “goal” of AGI.

If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

replies(1): >>44496156 #
6. nomel ◴[] No.44496156{3}[source]
> No one is “requiring perfection”

> If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

Those sentences aren't compatible.

> but hallucination is a major issue

Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this.

> is in the opposite direction of the “goal” of AGI

This isn't a logical statement, so it's difficult to respond to. Hallucination isn't a direction that's being headed towards, it's being actively, with intent and $$$, headed away from.

replies(1): >>44497457 #
7. Velorivox ◴[] No.44496511[source]
> which are both completely irrational

Really!?

[0] https://i.imgur.com/ly5yk9h.png

replies(1): >>44499203 #
8. epidemian ◴[] No.44496702[source]
> Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists

What?? What does AGI have to do with this? (If this was some kind of hyperbolic joke, sorry, i didn't get it.)

But, more importantly, the GP only said that in a sane world, the ChatGPT creators should be the ones trying to fix this mistake on ChatGPT. After all, it's obviously a mistake on ChatGPT's part, right?

That was the main point of the GP post. It was not about "requiring perfection" or something like that. So please let's not attack a straw man.

replies(1): >>44497790 #
9. lucianbr ◴[] No.44497457{4}[source]
> Those sentences aren't compatible.

My web browser isn't perfect, but it does not hallucinate inexistent webpages. It sometimes crashes, it sometimes renders wrong, it has bugs and errors. It does not invent plausible-looking information.

There really is a lot middle gound between perfect and "accept anything we give you, no matter how huge the problems".

replies(2): >>44497901 #>>44499185 #
10. bravesoul2 ◴[] No.44497641[source]
There was demand for the problem. ChatGPT created demand for this solution.
11. nomel ◴[] No.44497790{3}[source]
> What does AGI have to do with this?

Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism. That requirement fundamentally, by the non-deterministic statistics that these run on, means you cannot use an LLM, as they exist today. They're not "truth" machines - use a database instead.

Saying "I don't know", with determinism is only slightly different than saying "I know" with determinism, since it requires being fully aware of what you do know, not at a fact level, but at a conceptual/abstract level. Once you have a system that fully reasons about concepts, is self aware of its own knowledge, and can find the fundamental "truth" to answer a question with determinism, you have something indistinguishable from AGI.

Of course, there's a terrible hell that lives between those two, in the form of: "Error: Question outside of known questions." I think a better alternative to this hell would be a breakthrough that allowed "confidence" to be quantified. So, accept that hallucinations will exist, but present uncertainty to the user.

[1] https://news.ycombinator.com/item?id=44496098

replies(2): >>44498919 #>>44501513 #
12. jeffhuys ◴[] No.44497901{5}[source]
Different tech, different failure modes.

> it sometimes renders wrong

Is close to equivalent.

13. JimDabell ◴[] No.44498195[source]
You sound like all the naysayers when Wikipedia was new. Did you know anybody can go onto Wikipedia and edit a page to add a lie‽ How can you possibly trust what you read on there‽ Do you think Wikipedia should issue groveling apologies every time it happens?

Meanwhile, sensible people have concluded that, even though it isn’t perfect, Wikipedia is still very, very useful – despite the possibility of being misled occasionally.

replies(2): >>44498212 #>>44498751 #
14. fzeroracer ◴[] No.44498212[source]
OK, so how do I edit ChatGPT so it stops lying then?
replies(1): >>44498235 #
15. JimDabell ◴[] No.44498235{3}[source]
You have ignored my point.

A technology can be extremely useful despite not being perfect. Failure cases can be taken into consideration rationally without turning it into a moral panic.

You have no ability to edit Wikipedia to stop it from lying. Somebody can come along and re-add the lie a millisecond later.

replies(2): >>44498251 #>>44498712 #
16. fzeroracer ◴[] No.44498251{4}[source]
No, I didn't ignore your point. I invalidated it because you're comparing apples to oranges.

And yes, I do have the ability to edit Wikipedia. Anyone can edit Wikipedia. I can go on any page I want, right now, and make changes to the page. If someone readds a lie, then eventually we can hit consensus as other editors enter the discussion. Wikipedia's basis is formed by consensus and controlled by individuals like you and I.

ChatGPT is not. It is controlled by one company; I cannot go edit the weights of ChatGPT to prevent it from lying about my app or anything else I do. I can only petition them to change it and hope that either I have enough clout or have a legal basis to do so.

replies(1): >>44498284 #
17. JimDabell ◴[] No.44498284{5}[source]
You’ve changed your position. This is what you originally said:

> how do I edit ChatGPT so it stops lying then?

This is what you have changed to:

> And yes, I do have the ability to edit Wikipedia.

You do not have the ability to edit Wikipedia so it stops lying, which is the relevant factor here.

18. latexr ◴[] No.44498712{4}[source]
No, that is not accurate. Wikipedia has a number of guardrails and such an edit war would be detected and investigated. Possibly the page protected and the offending IP or account banned.

Wikipedia edits are monitored and vandalism is taken seriously, especially on the more important pages.

replies(1): >>44507350 #
19. latexr ◴[] No.44498751[source]
> despite the possibility of being misled occasionally.

There is a chasm of difference between being misled occasionally (Wikipedia) and frequently (LLMs). I don’t think you understand how much effort goes on behind the scenes at Wikipedia. No, not everyone can edit every Wikipedia page willy-nilly. Pages for major political figures often can only be edited with an account. IPs like those of iCloud Private Relay are banned and can’t anonymously edit the most basic of pages.

Furthermore, Wikipedia was always honest about what it is from the start. They managed expectations, underpromised and overdelivered. The bozos releasing LLMs talk about them as if they created the embryo of god, and giving money to their religion will solve all your problems.

replies(2): >>44507330 #>>44516434 #
20. penteract ◴[] No.44498919{4}[source]
You have a very strong definition of AGI. "Never being wrong" is something that humans fall far short of.
replies(1): >>44505243 #
21. tucnak ◴[] No.44499185{5}[source]
> It does not invent plausible-looking information.

This is where your analogy is falling apart; of course web browsers do not "invent plausible-looking information" because they don't invent anything in the first place! Web browsers represent a distinct set of capabilities, and as you correctly pointed out, these are often riddled with bugs and errors. If I was making a browser analogy, I would point towards fingerprinting; most browsers reveal too much information about any given user and system, either via cross-site cookies, GPU prints, and whatnot. This is an actual example where "ethics flew out the window long ago."

As the adjacent commenter pointed out: different software, different failure modes.

22. tucnak ◴[] No.44499203{3}[source]
Your screenshot is conveniently omitting the disclaimer below: "AI responses may include mistakes. Learn more[1]"

[1]: https://support.google.com/websearch/answer/14901683

replies(1): >>44499908 #
23. Velorivox ◴[] No.44499908{4}[source]
It isn't doing anything "conveniently", I was not shown the disclaimer (nor anything else, I assume it mostly failed to load).

In any case, if you really believe a disclaimer makes it okay for Google to display blatant misinformation in a first-party capacity, we have little to discuss.

replies(1): >>44500306 #
24. tucnak ◴[] No.44500306{5}[source]
https://www.google.com/search?q=is+all+of+oregon+north+of+ne...

Show more -> Disclaimer and the feedback buttons are shown at the end. If you bothered enough to read the full response, you would have seen the disclaimer, but you never did, so you haven't. For something to be considered "misinformation," in the very least the subject of speech has to be asserting its truthfulness—and, indeed, Google makes no such claims. The claim they're making is precisely that its search result-embedded "[..] responses may include mistakes." In this specific case, they are not asserting truthfulness.

FWIW, Gemini 2.5 Pro answers the question correctly.

The search hints are clearly a low-compute first approximation, which is probably correct for most trivial questions which is probably the majority of user queries, and it's not surprising that it fails in this specific instance. The application doesn't allow for reasoning due to scale; even Google cannot afford to run reasoning traces on every search question. I concur that there's very little to discuss: you seemingly made up your mind re: LLM technology, and I doubt you will appreciate the breaking-up of your semantics to begin with.

replies(1): >>44502060 #
25. ◴[] No.44500852[source]
26. epidemian ◴[] No.44501513{4}[source]
> If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then [...]

Ok, sure. But why would you choose to define hallucinations in a way that is contrary to common sense and the normal understanding of what an AI hallucination is?

The common definition of hallucinations is basically: when AI makes shit up and presents it as fact. (And the more technical definition also basically aligns with that.)

No one would say that if the AI takes the data you provide in the prompt and can deduce a correct answer for that specific data —something that is not directly or indirectly present in its training data— it would be hallucinating. In fact that would be an expected thing for an intelligent system to do.

It seems to me you're trying to discuss with something nobody said. You're making it seem that saying "it's bad that LLMs can invent wrong/misleading information like this and present it as fact, and that the companies that deploy them don't seem to care" is equivalent to "i want LLMs to be perfect and have no bugs whatsoever", and then discuss about how ridiculous is to state the latter.

replies(2): >>44505262 #>>44505280 #
27. ◴[] No.44502060{6}[source]
28. rustyminnow ◴[] No.44502343{3}[source]
If I were someone who ran ChatGPT then yeah, I'd have something to improve. But most of us aren't, and can't do anything about it - so might as well make lemons into lemonade instead of getting hung up on the obvious and unchangeable fact that our future is filled with AI garbage.
29. nomel ◴[] No.44505243{5}[source]
That's not my definition of AGI. To simplify what I said, "never being wrong" (aka "don't hallucinate") requires a level of agency and rigor that could only be achieved by something that would be an AGI. I said "determinism would require an AGI", not "AGI are deterministic".

Note that "never being wrong" can also be achieved by an "I looked into it, and there's no clear answer.", which is the correct answer for many questions (humans not required).

30. ◴[] No.44505262{5}[source]
31. nomel ◴[] No.44505280{5}[source]
I intentionally referenced their comment to make it clear, to you, that perfection is their twice stated requirement, even with a charitable interpretation. Here they are again:

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again

> If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

Neither of those ("didn't happen again" and "don't hallucinate") are logically ambiguous or flexible. I can only respond to what they wrote.

32. rbits ◴[] No.44505736[source]
Sure, but this isn't new. LLMs have been doing this for ages. That's why people aren't talking about it as much.
33. JimDabell ◴[] No.44507330{3}[source]
> I don’t think you understand how much effort goes on behind the scenes at Wikipedia.

I understand Wikipedia puts effort in, but it’s irrelevant. As a user, you can never be sure that what you are reading on Wikipedia is the truth. There are good reasons to assume that certain topics are more safe and certain topics are less safe, but there are no guarantees. The same is true of AI.

> Wikipedia was always honest about what it is from the start.

Every mainstream AI chatbot includes wording like “ChatGPT can make mistakes. Check important info.”

34. JimDabell ◴[] No.44507350{5}[source]
> such an edit war would be detected and investigated.

As far as I understand Wikipedia’s rules, no it wouldn’t. Wikipedia considers it an edit war when there are three reverts in a 24 hour period. So if you remove a lie and somebody reverts that change, that is not considered an edit war. If they do it twice, it’s still not an edit war. If they do it three times, but over a greater time period than a single day, it’s still not an edit war. If somebody else re-adds the lie, it’s not an edit war either.

Regardless, it’s not important. The point I am making is that as a user, you cannot be sure that Wikipedia is not serving up lies to you.

35. joegibbs ◴[] No.44516434{3}[source]
20 years ago though, I think our teachers had the right idea when they said Wikipedia wasn't a reliable source and couldn't be counted. It's much better these days but I checked an old revision (the article on 9/11) the other day and barely anything was sourced, there were parts written in first person, lots of emotive language.