AI assistants misrepresent news content 45% of the time

Now let's run this experiment against the editorial boards in newsrooms.

Obviously, AI isn't an improvement, but people who blindly trust the news have always been credulous rubes. It's just that the alternative is being completely ignorant of the worldviews of everyone around you.

Peer-reviewed science is as close as we can get to good consensus and there's a lot of reasons this doesn't work for reporting.

> Now let's run this experiment against the editorial boards in newsrooms.

Or against people in general.

It's a pet peeve of mine that we get these kinds of articles without a baseline established of how people do on the same measure.

Is misrepresenting news content 45% of the time better or worse than the average person? I don't know.

By extension: Would a person using an AI assistant misrepresent news more or less after having read a summary of the news provided by an AI assistant? I don't know that either.

When they have a "Why this distortion matters" section, those things matter. They've not established if this will make things better or worse.

(the cynic in me want another question answered too: How often does reporters misrepresent the news? Would it be better or worse if AI reviewed the facts and presented them vs. letting reporters do it? again: no idea)

The difference is the ease with which AI can be rolled out, scaled up, and woven into the fabric of our interactions with society.

That makes understanding the baseline all the more important. It could be a disaster, or it could in fact be a distinct improvement. Every time someone pushes a breathless headline about failure rates of AI without comparing it to a human baseline, they are in essence potentially misleading us because without that baseline we don't know whether it's better or worse.