Trains cancelled over fake bridge collapse image

I think we’re just getting started, with fake images and videos.

I suspect that people will be killed, because of outrage over fake stuff. Before the Ukraine invasion, some of the folks in Donbas made a fake bomb, complete with corpses from a morgue (with autopsy scars)[0]. That didn’t require any AI at all.

We can expect videos of unpopular minorities, doing horrible things, politicians saying stuff they never said, and evidence submitted to trial, that was completely made from whole cloth.

It’s gonna suck.

[0] https://www.bellingcat.com/news/2022/02/28/exploiting-cadave...

So far, I see the most concern about this sort of thing from people who came of age around or after Web 2.0 hit, at a time when even a good photoshop wasn’t too hard to place as fake.

Those I know who lived through this issue when digital editing really became cheap seem to be more sanguine about it, while the younger generation on the opposite side side is some combination “whatever” or frustrated but accept that yet another of countless weird things has invade a reality that was never quite right to begin with.

The folks in between, I’d say about the 20 years from age 20 to 40, are the most annoyed though. The eye of the storm on the way to proving that cyberpunk lacked the required imagination to properly calibrate our sense of when things were going to really get insane.

> We can expect videos of unpopular minorities, doing horrible things

Expect? You can post a random image of an unpopular minority, add some caption saying they did horrible things, that is not reflected in the image at all, and tons of people will pile on. Don’t even need a fake video.

In fact, this is already happening on a daily basis.

IA ain't the problem here, so called social media are.

For some reason this hurts worse.

I was listening to James O’Brien on LBC, and [IIRC] he said he was serving jury duty with a woman who was convinced that Volodymyr Zelenskyy had spent hundreds of million of dollars on a super-yacht.

He asked if she had any evidence for that claim, and she produced a picture of a boat.

He said “That’s just a picture of a boat.”

Well, some news organizations are more than willing to spread fake news as well, so it's hardly limited to social media. I think it's just in-group vs out-group mentality, and a need to hate.

Ironically, there is no evidence that woman ever said that.

Oh heck yes. One India focused study that I saw, introduced me to the term Cheap Fakes. Another report studied how genAI made phishing pipelines more efficient, allowing profitable targeting of groups who hitherto were too poor to be targetted.

So on one end you have large scale pollution of the information commons, and on the other end we are now creating predator pipelines to generate content with all the efficiency of our vaunted AI productivity. Its creating a dark forest for normal people to navigate, driving more government efforts to bring control. This in turn puts this in conflict with freedom of speech and expression while dovetailing nicely with authoritarian tendencies.

Yes, Its heartening to hear all the people who find productivity gains from AI, but in totality it feels like we got our wishes granted by the Evil Genie.

There is, in fact, evidence that hundreds if not thousands of random people have said that: https://xcancel.com/KimDotcom/status/1729171832430027144

Perhaps you could even find that specific woman leaving an outraged comment over photos of boats if you looked hard enough!

It was not happening on a daily basis on Twitter before Elon Musk. The endless flow of racism and bigotry on that website is a choice.

It's convenient to blame the amorphous thing "social media" instead of the actual people responsible. There are only a handful of them: Elon Musk, Mark Zuckerberg, etc.

And stopping it is simple. It's a choice.

It was, but it wasn't pushed into everyone's filter bubble.

Hah, Kim Dotcom is still around? In the 90s he was bragging that he's this super hacker that made millions, his website posted pics of parties, cars, girls, and yachts, and it turned out those were bought/rented using swindled investor money (ironic that he's accusing Zelensky of the same crime). Then he became a sort of hero when the US/NZ governments Team 6-ed his house for the crime of aiding copyright infringement.

Now he's a Putin/Trump apologist...

Yes, but in that story, parent only has the word of that Journalist. I personally don't even have that, I only have a post about it.

My deeper point is that it's arguably very difficult to establish a global, socially acceptable lower threshold of trust. Parent's level is, apparently, the word of a famous Journalist in a radio broadcast. For some, the form of a message alone makes the message worthy of trust, and AI will mess with this so much.

Who cares about a single or two Yachts. Ukraine likely made 100 billion USD disappear and there were many people expecting just that. Just like some of the "donated equipment" started showing up on all sorts of black markets once it was shipped to Ukraine. It's just the obviously controlled media in Europe that stopped mentioning Ukraine's corruption issues right after February 2022.

Obviously I can only be a Putin-loving propaganda bot for saying such things.

> We can expect videos of unpopular minorities, doing horrible things

While manipulation of photos exist, and real photos misattributed are very common, for the most part a lot of that does happen as well. And some people are too quick to ignore or gloss over it

Whether you trust the word of the journalist has little relation to the story. The "socially acceptable lower threshold of trust" is not static for all stories; it changes depending on the stakes of the story.

Non-consequential: A photo of a cat with a funny caption. I am likely to trust the caption by default, because the energy of doubting it is not worth the stakes. If the caption is a lie, it does nothing to change my worldview or any actions I will ever take. Nobody's life will be worse off for not having spent an hour debunking an amusing story fabricated over a cat photo.

Trivially consequential: Somebody relates a story about an anonymous, random person peddling misinformation based on photos with false captions on the internet. Whether I believe that specific random person did has no bearing on anything. The factor from the story that might influence your worldview is the knowledge that there are people in the world who are so easily swayed by false captions on photos, and that itself is a trivially verifiable fact, including other people consuming the exact photo and misinformation from the story.

More consequential: Somebody makes an accusation against a world leader. This has the potential to sway opinions of many people, feeding into political decisions and international relations. The stakes are higher. It is therefore prudent not to trust without evidence of the specific accusation at hand. Providence of evidence does also matter; not everything can be concretely proven beyond a shadow of a doubt. We should not trust people blindly, but people who have a history of telling the truth are more credible than people who have a history of lying, which can influence what evidence is sufficient to reach a socially acceptable threshold of trust.

>We can expect videos of unpopular minorities, doing horrible things, politicians saying stuff they never said, and evidence submitted to trial, that was completely made from whole cloth.

AI videos of unpopular minorities already comprise an entire genre and AI political misinformation is already mainstream. I'm pretty sure every video of Donald Trump released by the WH is AI generated, to make him look less senile and frail than he really is. We're already there.

Everybody is aware the Ukraine has major corruption issues. It is frequently covered in the media and is common knowledge.

I have no doubt however that Europe (and hopefully the wider world) is less worried about that corruption than they are about Russian military aggression. And there will be some level of media focus on that – rightly so, where the focus should be on grinding the Russian kleptostate into dust as quickly and thoroughly as possible.

You're not a propaganda bot; you're just making their lives easier.

Corruption in Ukraine is constantly in the news. https://www.economist.com/search?q=ukraine+corruption&nav_so...

The point about the stakes is a good one. But there is an individiual factor to it. And maybe it's exactly because of the stakes you mention: if you perceive your personal stakes to be low, or might even gain something out of redistributing the message, no matter if fabricated or not, your threshold might be low as well.

If you're happy with your tax euros disappearing in Ukraine, good for you.

I know for a fact via family ties that major newsrooms in Germany received instructions to tune out the corruption angle once the war started. I'm sure it's all nothing though and that Putin will find himself in Poland next year. Of course!

What's your point though? There's corruption in Ukraine. Ok.

There's corruption in your country too, do you refuse to pay taxes? Or do you still pay them because some good comes with the little bad? Same deal.

Where does the corruption come from?

It comes from an old culture that Ukraine is trying to remove themselves from, hence the large amount of corruption charges we see.

The same culture is incidentally what makes Russia one of the most corrupt countries in the world.

People were able to make very realistic fakes of anything 10-20 years ago, using basic tools. Just ask the UFO nuts or the NSFW media enthusiasts. And like what you mentioned, staged scenes have become somewhat common as well, including before the internet.

We can expect more of the same. Random unverified photo and video should not be trusted, not in 2005, not in 2015, and not today.

I believe that this "everything was fine but it's going to get really bad" narrative is just yet another attempt at regulatory capture, to outlaw open-source AI. This entire fake bridge collapse might very well be a false flag to scare senile regulators.

If sending hundreds of billions of tax payer money to a known oligarch run cleptocracy is comparable to some German conservative party affiliate making a couple of millions using shady COVID mask deals is comparable to you, I rest my case.

It's all corruption in the end so who cares, right?

Two things can be true at the same time - we don't want Russia to absorb Ukraine and then further threaten the eastern border of the EU, and we don't want Ukraine to be corrupt.

> > Trivially consequential: Somebody relates a story about an anonymous, random person peddling misinformation based on photos with false captions on the internet. Whether I believe that specific random person did has no bearing on anything.

> The point about the stakes is a good one. But there is an individiual factor to it.

Indeed. The so called "trivially consequential" depends on whether you're the person being "mis-informationed" about or not. You could be a black man with a white grandchild, and someone could then take a video your wife posted of you playing with your grandchild, and redistribute it calling you a pedophile, causing impact to your life and employment. Those consequences don't seem trivial to the people impacted.

True story: https://www.theguardian.com/world/2025/aug/20/family-in-fear...

And in the Ukraine we see that the corruption is uncovered punished, even if it is in the direct circles of the president.

There are problems in uncovering it, but the attempt to get rid of corruption is a big factor in the whole situation and one of the things Russia fears.

For Russia a corrupt system was a lot simpler to influence and Ukraine showing how a partially Russian speaking country, where people moved back and forth, fighting corruption was a threat to the system.

This is a complete and total misrepresentation of what I said. The key point here is that the "accused" in the trivial story is anonymous. They are fungible. Their identity is irrelevant to the story; it is merely an anecdote about the fact that a person like this exists, and people who exhibit the exact same behaviour as them verifiably do exist, so there is nothing to be misinformed about. A tangible accusation against a specific individual is completely different, and obviously is consequential.

In my opinion, this isn't a problem of AI. the people who get deceived by this are willing participants in the lie. When proven wrong, they will fall back to the echo chamber and rely on it to give them more false facts. They won't seek information outside of their own circle. They cannot be understood as merely passively misinformed. They are actively lying to themselves.

Still around and at a huge cost to the New Zealand taxpayer - people used to have some sympathy but public opinion turned against him years ago. His extradition was declared ok, long overdue he was put on a plane and made somebody else's problem.

this is true, my dad is Volodymyr Zelensky

> to tune out the corruption angle once the war started

Oh man, wait until you hear about what’s going on in the US, we’re experiencing corruption to a degree you can’t even imagine.

I'm sure this will be exactly the popular attitude - yes, the evidence and videos people see and form emotional reactions based on are fake, but the problem is real and so we should let it slide and just assume something exactly like the AI video happened anyway.

> You can post a random image of an unpopular minority, add some caption saying they did horrible things, that is not reflected in the image at all, and tons of people will pile on.

We call this journalism and this is a respectable profession. /s

What people tend to forget or dispel is that everything in society is based on trust. You can of course try to test every new fact for soundness with all the facts you already know, but fundamentally you need to trust something and in the end it just becomes a function of how much you trust the messenger. Undoing trustworthiness is a big issue and will lead to a lot of unrest in society.

The reason you are not murdered today is not, because murder is hardly punished or hard to do, it is because most people aren't murders. If they were, we wouldn't be able to suppress it with force, we would simply live in hell.

Motivated people (nation states) were able to do this even hundred years ago. The issue is simply that most people didn't did that.

> One India focused study that I saw, introduced me to the term Cheap Fakes

Wasn't that term entirely invented by the Democratic party to dismiss videos of Biden's "senior moments"?

I'm curious if the term predates that or maybe you're not in the US?

In my family it's the other way around - it's the people that used to tell us not to talk to strangers on the internet, and not to believe everything we see on the internet, who are now doing precisely that.

What you'll tend to notice with "willing participants" is that they're not looking for truth, they're looking for confirmation. No-one asks for proof when you tell them what they want to hear.

I heard it this year, but its a term that once you hear it, you "get it". Definitely not an India only thing.

Ukraine was, and is still, one of the most corrupt developed countries in the world. Whether it is slightly more or slightly less corrupt than Russia I do not know. Both are Oligarchic in nature. In my opinion one of the reasons the various peace deals have not succeeded yet is because they fail to acknowledge the Oligarchic nature of both states and that they will both need to continue in that mode going forwards, probably as a frozen conflict or in a system where it is in neither interest to disrupt the balance (because it would end the corruption, pocket-lining, theft, etc). Of course for the ordinary unfortunate Ukrainian, well he/she matters little or not at all to rulers on either side.

The people of New Zealand should be mad at the illegal tactics used by the FBI and GCSB. And why should he be extradicted to a country he never visited?

>What people tend to forget or dispel is that everything in society is based on trust

Trust seems to have been completely erroded on the internet. Majority of the mainstream sites feed users "news" posts from unverified unknown accounts. Its bad, we need a way to get back a base level of trust.

But a fake video has a much greater effect that can even prompt companies or gov orgs to take action in response (as in this case).

It seems to have been originated by two authors of a book on the 2024 election, where manipulative fake and misleading videos by the GOP were a widespread issue.

Not sure why you're trying to pretend that the idea of fake videos is some anti-GOP conspiracy.

Specifically the term "Cheap fakes" I have seen applied to genuine videos as well as doctored videos, all in response to videos of Biden though.