←back to thread

52 points anigbrowl | 9 comments | | HN request time: 0.213s | source | bottom
Show context
kylestanfield ◴[] No.44407748[source]
Realistic AI video generation will put us firmly in a post-truth era. If those videos were 25% more realistic it would be indistinguishable from a real interview.

The speed with which you’ll be able to create and disseminate propaganda is mind blowing. Imagine what a 3 letter agency could do with their level of resources.

replies(9): >>44407935 #>>44408117 #>>44408162 #>>44408294 #>>44408304 #>>44408347 #>>44408383 #>>44408399 #>>44410227 #
patrakov ◴[] No.44407935[source]
No, it won't.

Expected reaction: every camera manufacturer will embed chips that hold a private key used to sign and/or watermark photos and videos, thus attesting that the raw footage came from a real camera.

Now it only remains to solve the analog hole problem.

replies(4): >>44408000 #>>44408024 #>>44408046 #>>44408263 #
1. hansvm ◴[] No.44408263[source]
- Attach a screen to the camera. Bonus points for bothering to calibrate that contraption.

- Watermarking is nearly useless as a way of conveying that information, either visibly distorting the image or being sensitive to all manner of normal alterations like cropping, lightness adjustments, and screenshotting.

- New file formats are hard to push to a wide enough audience for this to have the desired effect. If half the real images you see aren't signed, ignoring the signature becomes second-nature.

- Hardware keys can always be extracted in O(N) for an N-bit key. The constant factor is large, but not enough to deter a well-funded adversary. The ability to convincingly fake, e.g., video proof you weren't at a crime scene would be valuable in a hurry. I don't know the limits, but it's more than the 2-10 million dollars you need to extract a key.

- You mentioned the analog hole problem, but that's also very real. If the sensor is engineered as a separate unit, it's trivial to sign whatever data you want. That's hard to work around because camera sensors are big and crude, so integration a non-removable crypto enclave onto one is already a nontrivial engineering challenge.

- If this doesn't function something like TLS with certificate transparency logs and chains of trust then one compromised key from any manufacturer kills the whole thing. Would the US even trust Chinese-signed images? Vice versa? The government you obey has a lot of power to steal that secret without the outside world knowing.

- Even if you do have CT logs and trust the company publishing to them to not publish compromised certs, a breach is much worse than for something like TLS. People's devices are effectively just bricked (going back to that 3rd point -- if all the images you personally take aren't appropriately signed, will a lack of signing seem like a big deal?). If you can update the secure enclave then an adversary can too, and if updating tries to protect itself by, e.g., only sending signed bytecode then you still have the problem that the upstream key is (potentially) compromised.

- Everyone's current devices are immediately obsolete, which will kill adoption. If you grandfather the idea in, there's still a period of years where people get used to real images not being signed, and you still have a ton of wasted money and resources that'll get pushback.

Etc. It's really not an easy problem.

replies(1): >>44408438 #
2. AlotOfReading ◴[] No.44408438[source]
Persistent media watermarking through the analog hole is a solved problem and has been for years. It's standard practice on films.

What does it even mean that hardware keys are extractable in O(N) time? If there's some reasonable multiple of N where you can figure out a key, your cryptosystem is broken, physical or not.

It's also very straightforward to attach metadata to media and wouldn't take a format change.

replies(3): >>44408582 #>>44408655 #>>44409683 #
3. perching_aix ◴[] No.44408582[source]
> Persistent media watermarking through the analog hole is a solved problem and has been for years. It's standard practice on films.

Can you expand on that a bit? Wikipedia's coverage on this seems mostly historical and copy protection focused.

replies(1): >>44409146 #
4. card_zero ◴[] No.44408655[source]
The problem would be spurious watermarks, not vanishing ones. Create fake video, point camera at screen, re-record it. Now it's fake and authenticated as a genuine camera recording.
5. AlotOfReading ◴[] No.44409146{3}[source]
The basic idea is that you apply a very, very large amount of error correction to the tag and inject it into the media so that enough survives the severe geometric, color, and luminance distortions of a camcorder to recover the data out the other end. You then download the pirated cams and sue the theater.

There's a fair bit of public information out there on theoretical techniques (e.g. https://www.intechopen.com/chapters/71851), but I'm not deeply familiar with what's actually used in industry, for example by imatag.

replies(1): >>44409479 #
6. perching_aix ◴[] No.44409479{4}[source]
Interesting, and the paper is surprisingly accessible to read as well, thanks.

One critique I can lodge against this is that to me it reads like the security model in this scheme trusts the venue to not tamper with the projection equipment. This may not map well to everyday camera recording situations, where the camera owner / operator may have a vested interest and capability in tampering with the camera itself.

replies(1): >>44409668 #
7. AlotOfReading ◴[] No.44409668{5}[source]
There's a fair bit of protection for the projection cameras. They're actually always-connected devices that get streaming permissions from a remote server before starting a show, device roots of trust, tamper detection systems, and so on. The movie file is essentially encrypted at rest until showtime. Plus the techs are generally not that technically advanced, and theaters face the threat of lawsuits/never being allowed to show movies again if they breach their contractual obligations to reverse engineer the equipment. It's generally effective.

The point isn't to completely, 100% close the analog hole here (and at least one piracy group seems to have leaks of raw movie data despite all of this security), but it's effective at making compliance the least costly option for almost everyone involved worldwide. It's one of the major reasons we've seen a shift by pirate groups to preproduction leaks or alternative methods.

8. hansvm ◴[] No.44409683[source]
> cryptosystem is broken

The problem isn't the cryptography. It's the hard part of cryptography, key management. In this case, the key is in the hands of the attacker, written down for all to see, and the only real defense is that it's written in a tiny "font" next to a bunch of other data.

You're right, that's a broken model. That's exactly what I'm saying. It's kind of like the current craze of using a phone number on login for "security" -- doing so increases the cost to an attacker by a fixed amount ($10-$15 or so) and is only reasonable for security (as opposed to selling out to data brokers) if the value to the attacker of the account in question is less than that. The security doesn't come from actual impossibility guarantees but by trying to stack a big enough constant factor onto the thing that nobody actually cares to break it.

> metadata vs file format change

Ah, exif and all the other slightly underspecified and incompatible metadata formats that no vendors fully agree on and are stripped by various middlemen and not relevant when, e.g., iphones auto-lossily-compress the formerly signed images when emailing and whatnot. We _could_ use those, but I don't think the software work involved to actually make that technosocial system work correctly is less than a new file format (e.g., with progressive rendering and something conceptually equivalent to signing each progression to account for compression use-cases).

> persistent media watermarking

Last I checked, failure rates, both type I and type II, are in the single-digit percentage ranges or worse, and that's with very low bit rates. Is the SOTA much better than that now?

replies(1): >>44410028 #
9. AlotOfReading ◴[] No.44410028{3}[source]
Physical extraction doesn't seem like an O(N) thing to me, but granted.

Honestly, couldn't tell you what the failure rates are for SOTA in the real world as I don't work in that industry and numbers about anything DRM-related are kept quiet. Single digit error rates would be fantastically useful to the kind of person that thinks that's a good technology though, and it's usually pretty tunable.