FFmpeg to Google: Fund us or stop sending bugs

> "Don't come to me with problems, come with solutions"

The problem is, the issue in the article is explicitly named as "CVE slop", so if the patch is of the same quality, it might require quite some work anyway.

The linked report seems to me to be the furthest thing from "slop". It is an S-tier bug report that includes a complete narrative, crash artifacts, and detailed repro instructions. I can't believe anyone is complaining about what is tied for the best bug report I have ever seen. https://issuetracker.google.com/issues/440183164?pli=1

It's a good quality bug report.

But it's also a bug report about the decoder for "SANM ANIM v0" - a format so obscure almost all the search results are the bug report itself. Possibly a format exclusive to mid-1990s LucasArts games [1]

Pretty crazy that ffmpeg supports the codec in the first place, IMHO.

I can understand volunteers not wanting to sink time into maintaining a codec to play a video format that hasn't been used since the Clinton administration. gstreamer divides their plugins into 'good', 'bad' and 'ugly' to give them somewhere to stash unmaintained codecs.

[1] https://web.archive.org/web/20250419105551/https://wiki.mult...

It's a codec that is enabled by default at least on major Linux distributions, and that will be processed by ffmpeg without any extra flags. Anyone playing an untrusted video file without explictly overriding the codec autodetection is vulnerable.

The format being obscure and having no real usage doesn't help when it's the attackers creating the files. The obscure formats are exposing just as much attack surface as the common ones.

> Pretty crazy that ffmpeg supports the codec in the first place, IMHO.

Yes.

Yeah but as you can see from the bug report ffmpeg automatically triggers the codec based on file magic, so it is possible that if you run some kind of network service or anything that handles hostile data an attacker could trigger the bug.

Ffmpeg makes it trivial to enable and disable individual codecs at compile time. Perhaps it's the Linux distros that need to make a change here?

Every change breaks somebody's workflow.

Sure, it's a valid bug report. But I don't understand why there has been so much drama over this when all the ffmpeg folks have to do is say "sorry, this isn't a priority for us so we'll get to it as soon as we can" and put the issue in the backlog as a low priority. If Google wants the issue fixed faster, they can submit a fix. If they don't care enough to do that, they can wait. No big deal either way. Instead, ffmpeg is getting into a public tiff with them over what seems to be a very easily handled issue.

I get that the ffmpeg people have limited time and resources, I get that it would be nice if Google (or literally anyone else) patched this themselves and submitted that upstream. But "everyone else down stream of us should compile out our security hole" is a terrible way to go about things. If this is so obscure of a bug that there's no real risk, then there's no need for anyone to worry that the bug has been reported and will be publicized. On the other hand, if it's so dangerous that everyone should be rebuilding ffmpeg from source and compiling it out, then it really needs to be fixed in the up stream.

Edit: And also, how is anyone supposed to know they should compile the codec our unless someone makes a bug report and makes it public in the first place?

There are dozens if not hundreds of issues just like this one in ffmpeg, except for codecs that are infinitely more common. Google has been running all sorts of fuzzers against ffmpeg for over a decade at this point and it just never ends. It's a 20 year old C project maintained by poorly funded volunteers that mostly gives every media file ever the be-liberal-in-what-you-accept treatment, because people complain if it doesn't decode some bizarrely non-standard MPEG4 variant recorded with some Chinese plastic toy from 2008. Of course it has all of the out-of-bounds bugs. I poked around on the issue tracker for like 5 minutes and found several "high impact" issues similar to the one in TFA just from the last two or three months, including at least one that hasn't passed the 90 day disclosure window yet.

Nobody who takes security even remotely seriously should decode untrusted media files outside of a sandboxed environment. Modern media formats are in themselves so complex one starts wondering if they're actually Turing complete, and in ffmpeg the attack surface is effectively infinitely large.

The issue is CVE slop because it just doesn't matter if you consider the big picture.

Some example issues to illustrate my point:

https://issuetracker.google.com/issues/436511754 https://issuetracker.google.com/issues/445394503 https://issuetracker.google.com/issues/436510316 https://issuetracker.google.com/issues/433502298

FFMPEG is upset because Google made the exploit public. They preferred that it remained a zero-day until they decided it was a priority.

I don't understand how anyone believes that behavior is acceptable.

I don't get why you think linking to multiple legitimate and high quality bug reports with detailed analysis and precise reproduction instructions demonstrates "slop". It is the opposite.

This is software that is directly or indirectly run by millions of people on untrusted media files without sandboxing. It's not even that they don't care about security, it's that they're unaware that they should care. It should go without saying that they don't deserve to be hacked just because of that. Big companies doing tons of engineering work to add defense in depth for use cases on their own infrastructure (via sandboxing or disabling obsolete codecs) doesn't help those users. Finding and fixing the vulnerabilities does.

Yes, you're very right. They could simply have killed a codec that no one uses anymore. Or put it behind a compile flag, so if you really want, you can still enable it

But no. Intentionally or not, there was a whole drama created around it [1], with folks being criticized [2] for saying exactly what you said above, because their past (!) employers.

Instead of using the situation to highlight the need for more corporate funding for opensource projects in general, it became a public s**storm, with developers questioning their future contributions to projects. Shameful.

[1] https://news.ycombinator.com/item?id=45806269

[2] https://x.com/FFmpeg/status/1985334445357051931

Best response would be to drop this codec entirely, or have it off by default. At least distros should do that.

All of these reports are effectively autogenerated by Big Sleep from fuzzing.

Again, Google has been doing this sort of thing for over a decade and has found untold thousands of vulnerabilities like this one. It is not at all clear to me that their doing so has been all that valuable.

Google fuzzing open source projects has eliminated a lot of low hanging fruit from being exploited. I am surprised you think that finding these vulnerabilities so they can be fixed has not been valuable.

There's no exploit on the bug report at least, unless you consider the crash reproducer one.

I'm sure that a hacker wouldn't think of trying to use an obscure format...

https://googleprojectzero.blogspot.com/2021/12/a-deep-dive-i...

UAF bugs lead to RCE exploit chains.

It feels like maybe people do not realize that Google is not the only company that can run fuzzers against ffmpeg? Attackers are also highly incentivized to do so and they will not do you the courtesy of filing bug reports.

Hmmmm. There's probably just one guy who wrote the ffmpeg code for that format. _Maybe_ one or two more who contributed fixes or enhancements?

The ffmpeg project need to get in touch and get then to assign copyright to the ffmpeg project, then delete that format/decoder from ffmpeg. Then go back to Google with an offer to licence then a commercial version of ffmpeg with the fixed SANM ANIM v0 decoder, for the low low price of only 0.0001% of YouTube's revenue every year. That'd likely make them the best funded open source project ever, if they pulled it off.

That behaviour is indeed totally unacceptable. At your job. Where they're paying you, and especially if they're paying you at FAANG type pay scales.

If you're an unpaid volunteer? Yeah - nah. They can tell you "Sorry, I'm playing with my cat for the next 3 months, maybe I'll get to it after that?", or just "Fuck off, I don't care."

(I'm now playing out a revenge fantasy in my head where the ffmpeg team does nothing, and Facebook or Palantir or someone similar get _deeply_ hacked via the exploit Google published and theat starts the planets biggest ever pointless lawyers-chasing-the-deepest-pockets fight.)

> But "everyone else down stream of us should compile out our security hole" is a terrible way to go about things.

Is that somehow _less_ of a terrible way to think than "someone who's contributed their time as a volunteer to an open source software project that we have come to rely on, now has some sort of an obligation to drop everything and do more unpaid work for a trillion dollar company"?

> it really needs to be fixed in the up stream

Lots of people love using "Free Software" that they didn't have to write as essential parts of their business.

Way too many of them seem to blink right when they get to this bit of the licence they got it with:

SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

(That's directly from Section 15 "NO WARRANTY of https://code.ffmpeg.org/FFmpeg/FFmpeg/src/branch/release/4.0... )

> Google has been running all sorts of fuzzers against ffmpeg for over a decade at this point

Yeah. It's called YouTube... Why run fuzzers if you can get people to upload a few million random videos every day? ;-)

(I wonder if the BigSleep AI was trained on or prompted with YouTube error logs?)

  > If you used the scan to pdf functionality of a [Xerox] like this a decade ago, your PDF likely had a JBIG2 stream in it.

That's not an obscure format, that's an old format. Meanwhile with ffmpeg we're talking about

  > decoding LucasArts Smush codec, specifically the first 10-20 frames of Rebel Assault 2, a game from 1995.

That's both old and obscure.

Your point is still taken, but just to clarify that these are different situations. JBIG2 is included for legacy. The Lucas art codec is included for... completion's sake(?)

They can if someone manages to develop an exploit. Let's not confuse vulnerabilities and exploits.

The actual best response would be to run any "unsupported" codecs in a WASM sandbox. That way you are not throwing away work, Google can stop running fuzzers against random formats from 1995, and you can legitimately say that the worst that can happen with these formats is a process crash. Everybody wins.

> someone who's contributed their time as a volunteer to an open source software project that we have come to rely on, now has some sort of an obligation to drop everything and do more unpaid work for a trillion dollar company

If you could highlight the relevant part of the bug report that demanded the developers "drop everything" and do "unpaid work for a trillion dollar company", that would be great because I'm having trouble finding it. I see "hey, we found this bug, we found where we think the issue is in the code and here's a minimal reproduction. Also FYI we've sent you this bug report privately, but we will also be filing a public bug report after 90 days." And no, I don't think having a policy of doing a private bug report followed by a public report some time later qualifies as a demand. They could have just made a public report from the get go. They could also have made a private report and then surprised them with a public bug report some arbitrary amount of time later. Giving someone a private heads up before filing a public bug report is a courtesy, not a demand.

And it's really funny to complain about Google expecting "unpaid work for a trillion dollar company", when the maintainers proudly proclaim that the likes of no less than Google ARE paying them for consulting work on ffmpeg[1][2][3]

[1]: https://fflabs.eu [2]: https://fflabs.eu/about/ [3]: https://ffmpeg.org/consulting.html

Publicly posting an exploitable bug IS asking for someone to drop everything and come fix the issue NOW.

So when someone finds a bug in software, in your mind the only acceptable options are:

1) Fix it yourself

2) Sit on it silently until the maintainers finally get some time to fix it

That seems crazy to me. For one, not everyone who discovers a bug can fix it themselves. But also a patch doesn't fix it until it's merged. If filing a public bug report is expecting the maintainers to "drop everything and do free labor" then certainly dropping an unexpected PR with new code that makes heretofore unseen changes to a claimed security vulnerability must surely be a much stronger demand that the maintainers "drop everything" and do the "free labor" of validating the bug, validating the patch, merging the patch etc etc etc. So if the maintainers don't have time to patch a bug from a highly detailed bug report, they probably don't have time to review an unexpected patch for the same. So then what? Does people sit on that bug silently until someone finally gets around to having the time to review the PR. Or are they allowed to go public with the PR even though that's far more clearly a "demand to drop everything and come fix the issue NOW".

I for one am quite happy the guy who found the XZ backdoor went public before a fix was in place. And if tomorrow someone discovers that all Debian 13 releases have a vulnerable SSH installation that allows root logins with the password `12345`, I frankly don't give a damn how overworked the SSH or Debian maintainers are, I want them to go public with that information too so the rest of us can shut off our Debian servers.

Or perhaps you’re a FAANG security researcher and your time will be better spent serving the OSS community as a whole by submitting as many useful bug reports as possible, instead of slightly fewer reports with patches included.

In this particular case it’s hardly obvious which patch you should submit. You could fix this particular bug (and leave in place the horrible clunky codec that nobody ever uses) OR you could just submit a patch that puts it behind a compile flag. This is really a decision for the maintainers, and submitting the latter (much better!) patch would not save the maintainers any meaningful amount of time anyway.

This bug might lead to vulnerability and that's enough. It makes no sense to waste lot of time and research whether it is possible or not - it is faster to remove the buggy codec nobody needs or make a fix.

AI found the bug, but the analysis and bug report were entirely written by a human without AI assistance. Source: I work with the author.

xz was a fundamentally different problem, it was code that had been maliciously introduced to a widespread library and the corrupted version was in the process of being deployed to multiple distributions. The clock was very much ticking.

Responsible disclosure policies for contributor-driven projects can differ from commercial projects. Also, if Google has the funds to pay for bug finding, they also have the funds for bug fixing the community projects they depend on.

Here’s where I’m coming from: it would really suck if the outcome of all this was for ffmpeg to drop support for niche codecs.

It may be the case that ffmpeg cannot reasonably support every format while maintaining the same level of security. In that case, it makes sense for distros to disable some formats by default. I still think it’s great that they’re supported by the ffmpeg project.

I agree there would probably need to be some unified guidance about which formats to enable.

The problem is that if you have a process using ffmpeg and an attacker feeds it a video with this codec, ffmpeg will proceed to auto-detect the codec, attempt to decrypt and then then break everything.

If the format is old and obscure, and the implementation is broken, it shouldn't be on by default.

I don’t understand how it helps the community to publicly release instructions for attacking people, unless you’re trying to incentivize a company to fix their crap. In this case, there is no company to incentivize, so just report it privately.

You can say publicly that “there is an ABC class vulnerability in XYZ component” so that users are aware of the risk.

Anyone running this code with untrusted input needs to sandbox it (which Google has been doing all along).

It’s OSS so somebody who cares will fix it, and if nobody cares then it doesn’t really matter.

This also informs users that it’s not safe to use ffmpeg or software derived from it to open untrusted files, and perhaps most importantly releasing this tells the distro package maintainers to disable the particular codec when packaging.

Right, I just don’t see why they need to publish the actual exploit.

Sorry, I probably wasn't clear enough in my comment. I was trying to say that being old gives some legitimacy for existing. Just because it is old doesn't mean it isn't used. Though yes, this should be better determined to make sure it isn't breaking workflows you don't know about.

But old AND obscure, well it's nice that it is supported but enabled by default? Fully with you there.

They have not, neither have they indicated that they’re planning to do so.

The clock is always ticking. You have no idea when you find a vulnerability who knows about it or how or whether it is currently being actively exploited. A choice to delay disclosure is a choice to take on the risk that the bug is being actively exploited in order to reduce the gap (and risk in that gap) between public disclosure and remediations being available. But critically, it is a risk that is being forced on the users of the software. They are unable to make an informed decision about accepting the risk because they don't know there is a risk. Public disclosure, sooner rather than later MUST be the goal of all bug reports, no matter how serious and no matter how overworked the maintainers.

> Responsible disclosure policies for contributor-driven projects can differ from commercial projects.

The can, but there's not an obvious reason why they should. If anything, public disclosure timelines for commercial closed source projects should be much much longer than for contributor-driven projects because once a bug is public ANYONE can fix it in the contributor-driven project, where as for a commercial project, you're entirely at the mercy of the commercial entities timelines.

> Also, if Google has the funds to pay for bug finding, they also have the funds for bug fixing the community projects they depend on.

They do. And they do. They literally higher the ffmpeg maintainers via the maintainer's consulting business (fflabs.eu) and they routinely contribute code to the ffmpeg project.