←back to thread

421 points sohkamyung | 1 comments | | HN request time: 0.224s | source
Show context
scarmig ◴[] No.45669929[source]
If you dig into the actual report (I know, I know, how passe), you see how they get the numbers. Most of the errors are "sourcing issues": the AI assistant doesn't cite a claim, or it (shocking) cites Wikipedia instead of the BBC.

Other issues: the report doesn't even say which particular models it's querying [ETA: discovered they do list this in an appendix], aside from saying it's the consumer tier. And it leaves off Anthropic (in my experience, by far the best at this type of task), favoring Perplexity and (perplexingly) Copilot. The article also intermingles claims from the recent report and the one on research conducted a year ago, leaving out critical context that... things have changed.

This article contains significant issues.

replies(7): >>45669943 #>>45670942 #>>45671401 #>>45672311 #>>45672577 #>>45675250 #>>45679322 #
amarant ◴[] No.45672311[source]
Human journalists misrepresent the white paper 85% of the time.

With this in mind, 45% doesn't seem so bad anymore

replies(4): >>45672656 #>>45673221 #>>45673610 #>>45674322 #
ribosometronome ◴[] No.45673610[source]
Hell, human editors seem to misrepresent their journalists frequently enough that I'm left wondering if it's hyperbolic or not to guess if they misrepresent them 45% of the time, too.
replies(1): >>45676242 #
1. hinkley ◴[] No.45676242[source]
HN used to have a policy of not editorializing article titles when published here, but I've caught them modifying headlines a few times to match the source article instead of the linked article. One that stuck out was just the other day, and it had a confusing title that was not only wrong but also hard to parse.

Maybe we complained with enough concrete examples of how absolute shit editors and summarizers are now.