AI assistants misrepresent news content 45% of the time

Kagi News has been pretty accurate. Source information is provided along with the summary and key details too.

AI summarizes are good for getting a feel of if you want to read an article or not. Even with Kagi News I verify key facts myself.

What if the AI makes an interesting or important article sound like one you don't want to read? You'd never cross check the fact, and you'd never discover how wrong the AI was.

Or https://rawdiary.com

Integrity of words and author intent is important. I understand the intent of your hypothetical but I haven’t run into this issue in practice with Kagi News.

Never share information about an article you have not read. Likewise, never draw definitive conclusions from an article that is not of interest.

If you do not find a headline interesting, the take away is that you did not find the headline interesting. Nothing more, nothing less. You should read the key insights before dismissing an article entirely.

I can imagine AI summarizes being problematic for a class of people that do not cross check if an article is of value to them.

That's fair, but i also don't cross check news sources on average either. I should, but there in lies the real problem imo. Information is war these days, and we've not yet developed tools for wading through immense piles of subtly inaccurate or biased data.

We're in a weird time. It's always been like this, it's just much.. more, now. I'm not sure how we'll adapt.

agreed on Kagi News, and Particle News has been good, but they accepted funding from The Atlantic which evidently earns "Featured Article" positioning to articles from funding sources, muddying the clarity of biases, which Particle News has a nice graphic indicator for, though i've not seen it under promoted Feature Articles. Surely applies to other funding sources, but The Atlantic one was pretty recent.

There is more written material produced every hour than I could read in a lifetime, I am going to miss 99.9999% of everything no matter what I do. It's not like the headline+blurb you usually get is any better in this regard.

How do you verify a fact? Do you travel to the location and interview the locals? Or read scientific papers in various fields, including their own references, to validate summaries published by news sources? At some point you need to just trust that someone is telling the truth.

I’m pretty sure what the what your parent comment means is they verify that key facts outputted by the summary match what’s written in the source.

> I can imagine AI summarizes being problematic for a class of people that do not cross check if an article is of value to them.

I feel like that’s “the majority of people” or at least “a large enough group for it to be a societal problem”.

> Information is war these days

I don't know If i can agree with that. I think we make an error when we aggregate news in the way we do. We claim that "the right wing media" says something when a single outlet associated with the right says a thing, and vice versa. That's not how I enjoy reading the news. I have a couple of newspapers I like reading, and I follow the arguments they make. I don't agree with what they say half the time, but I enjoy their perspective. I get a sense of the "editorial personality" of the paper. When we aggregate the news, we don't get that sense, because there's no editorial. I think that makes the news poorer, and I think it makes people's views of what newspapers can be poorer.

The news shouldn't a stream of happenings. The newspaper is best when it's a coherent day-to-day conversation. Like a pen-pal you don't respond to.

I've has a similar experience with my own project that summarizes rss articles--the results have largely been pretty good, but I found using a "reasoning" model had much better results.

Kagi News is basically a summary of news articles fed into the context. It's different from what the op is about, that is just asking an LLM with web access to query the news.

I hate saying people are holding it wrong but given just given how LLMs work, how did anyone expect that this would go right? Managing the LLM's context is the game. I feel like ChatGPT has done such a disservice for teaching users how to actually use these tools and what their failure modes are.

It may help to set penalties for someone not telling the truth.