I mean sure, max performance is great if you control every part of your pipeline, but if you're accepting untrusted data from users-at-large ffmpeg has at least a half-dozen remotely exploitable CVEs a year. Better make sure your sandbox is tight.
https://ffmpeg.org/security.html
I feel like there's a middle ground where everyone works towards a secure and fast solution, rather than whatever position they've staked out here.
What's the alternative?
ffmpeg is a monopoly in the space which means that you either take the exact set of tradeoffs they offer, or... well, you have no alternatives, so take it.
Of course the alternatives are never going to be as good as the originals until they've had more effort put into them. It took _years_ until the Rust gzip/zip libraries surpassed the C ones while being more secure overall.