←back to thread

421 points sohkamyung | 1 comments | | HN request time: 0.264s | source
Show context
scarmig ◴[] No.45669929[source]
If you dig into the actual report (I know, I know, how passe), you see how they get the numbers. Most of the errors are "sourcing issues": the AI assistant doesn't cite a claim, or it (shocking) cites Wikipedia instead of the BBC.

Other issues: the report doesn't even say which particular models it's querying [ETA: discovered they do list this in an appendix], aside from saying it's the consumer tier. And it leaves off Anthropic (in my experience, by far the best at this type of task), favoring Perplexity and (perplexingly) Copilot. The article also intermingles claims from the recent report and the one on research conducted a year ago, leaving out critical context that... things have changed.

This article contains significant issues.

replies(7): >>45669943 #>>45670942 #>>45671401 #>>45672311 #>>45672577 #>>45675250 #>>45679322 #
1. happymellon ◴[] No.45679322[source]
You don't have to read very far to see the details.

> 45% of responses contained at least one meaningful error. Sourcing [...] is 31%, followed by accuracy 20%

And you can see the reason they think this is important on the second page just after the summary.

> More than 1 in 3 (35%) of UK adults instinctively agree the news source should be held responsible for errors in AI-generated news

So of course the BBC cares that Googles summary said that the BBC cites pornhub when talking about domestic abuse (when they didn't), because a large portion of people blame them for the fact that a significant amount of AI generated crap is wrong.