Netflix’s AV1 Journey: From Android to TVs and Beyond

I had forgotten about the film-grain extraction, which is a clever approach to a huge problem for compression.

But... did I miss it, or was there no mention of any tool to specify grain parameters up front? If you're shooting "clean" digital footage and you decide in post that you want to add grain, how do you convey the grain parameters to the encoder?

It would degrade your work and defeat some of the purpose of this clever scheme if you had to add fake grain to your original footage, feed the grainy footage to the encoder to have it analyzed for its characteristics and stripped out (inevitably degrading real image details at least a bit), and then have the grain re-added on delivery.

So you need a way to specify grain characteristics to the encoder directly, so clean footage can be delivered without degradation and grain applied to it upon rendering at the client.

Actual film grain (i.e., photochemical) is arguably a valid source of information. You can frame it as noise, but does provide additional information content that our visual system can work with.

Removing real film grain from content and then recreating it parametrically on the other side is not the same thing as directly encoding it. You are killing a lot of information. It is really hard to quantify exactly how we perceive this sort of information so it's easy to evade the consequences of screwing with it. Selling the Netflix board on an extra X megabits/s per streamer to keep genuine film grain that only 1% of the customers will notice is a non-starter.

Exactly. In the case of stuff shot on film, there's little to be done except increase bitrate if you want maximal fidelity.

In the case of fake grain that's added to modern footage, I'm calling out the absurdity of adding it, analyzing it, removing it, and putting yet another approximation of it back in.

> I'm calling out the absurdity of adding it, analyzing it, removing it, and putting yet another approximation of it back in

Why is it absurd? The entire encoding process is an approximation of the original image. Lossy compression inevitably throws away information to make the file size smaller. And the creation of the original video is entirely separated from the distribution of it. It'll be stored losslessly for one thing.

The only question that matters is does the image look good enough after encoding. If it doesn't look good enough then no one will watch it. If it does look good enough then you've got a viable video distribution service that minimizes its bandwidth costs.

"Lossy compression inevitably throws away information to make the file size smaller."

So? That fact only emphasizes the absurdity and loss potential here:

1. Acquire "clean" digital footage.

2. Add fake grain to said footage

3. Compress the grainy footage with lossy compression, wasting a bunch of the data on fake detail that you just added.

4. Analyze the footage to determine the character of the fake grain, calculate parameters to approximate it later with other fake grain

5. Strip out the fake grain, with potential for loss of some original image details

6. Re-add fake grain with the calculated parameters

If you don't see the absurdity there, I don't know what to tell you.