←back to thread

AI assistants misrepresent news content 45% of the time

(www.bbc.co.uk)
421 points sohkamyung | 2 comments | 22 Oct 25 13:39 UTC | HN request time: 0s | source
Show context
simonw ◴[22 Oct 25 14:46 UTC] No.45669931[source]▶
Page 10 onwards of this PDF shows concrete examples of the mistakes: https://www.bbc.co.uk/aboutthebbc/documents/news-integrity-i...

> ChatGPT / CBC / Is Türkiye in the EU?

> ChatGPT linked to a non-existent Wikipedia article on the “European Union Enlargement Goals for 2040”. In fact, there is no official EU policy under that name. The response hallucinates a URL but also, indirectly, an EU goal and policy.

replies(1): >>45670526 #
brabel ◴[22 Oct 25 15:21 UTC] No.45670526[source]▶
It did exist but got removed: https://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletio...

Quite an omission to not even check for that and it make me think that was done intentionally.

replies(2): >>45670612 #>>45671354 #
simonw ◴[22 Oct 25 16:13 UTC] No.45671354[source]▶
It's probably for the best that chat interfaces avoid making direct HTTP calls to sources at run-time to confirm that they don't 404 - imagine how much extra traffic that could add to an internet ecosystem which is suffering from badly written crawlers already.

(Not to mention plenty of sites have added robots.txt rules deliberately excluding known AI user-agents now.)

replies(1): >>45671648 #
1. magackame ◴[22 Oct 25 16:33 UTC] No.45671648[source]▶
Wouldn't it be the same amount of requests as a regular person researching something the old way?
replies(1): >>45672729 #
2. simonw ◴[22 Oct 25 17:53 UTC] No.45672729[source]▶
If you watch the thinking panel in ChatGPT with GPT-5 Thinking it often consults dozens of pages in response to a single prompt.