Mpv – A free, open-source, and cross-platform media player

I think technically he’s correct (I haven’t worked on media decoding code, but I understand how common video encoding formats work). If you have a long video with only a single key frame at the beginning then to step back you would need to, starting from the beginning of the video, decode every frame up to the previous frame you wanted to jump to in order to apply frame deltas, also assuming you have some sort of frame counter to determine when you’ve reached the target frame. In the worst case this does require a lot of compute, but this is an edge case if you primarily care about common video formats with normal encoding settings. I assume seeking backwards is also painfully slow on videos encoded in this manner, so why stepping back 1 frame is out of the question when compared to seeking backwards, I don’t fully understand, it must have something to do with precise frame counts being unavailable on some hardware decoders for some formats (and there being no good workaround) so you _may_ not actually go back 1 frame.

I don’t see any reason it couldn’t be supported for a set of formats with reasonable encoding/decoding settings, and provide some error message for other formats if a user attempts to step back, e.g. reverse frame stepping unavailable for current video due to format/encoding/decoding settings.

that's still a whole lot of yapping that as an end user I don't care about. i can frame scrub forwards and backwards in multiple other apps. right? very weird response from the vlc team in that original thread.

> video with only a single key frame at the beginning

I've literally never seen a video like that in my life, but I'd still expect it to work. Just decode everything starting from frame 1. My desktop can decode H265 at 1,666 fps. I can wait.

https://docs.nvidia.com/video-technologies/video-codec-sdk/1...

Wouldn’t creating in-memory key frames for every nth frame resolve substantial computations on a frame-by-frame basis?

> If you have a long video with only a single key frame at the beginning then

...you can't support the scrub bar efficiently either, so no one encodes video that way.

Typically to go to a frame you find the last IDR frame before it (and in reasonable encodings those are frequent enough) and decode forward until you get to the frame of interest. Doing that every time the user presses the single frame back button really doesn't seem that bad, and neither does holding onto some extra reference images for at least like 1080p frames. (8k video and such starts getting more expensive but maybe even then start doing all some references after the first press of the frame back button in this GOP or some such.)

It's of course work to do, and I'm not super motivated to send them that patch, and there's the question of it it would be merged and maintained indefinitely, but what folks are asking for is technically possible.

> I don’t see any reason it couldn’t be supported for a set of formats with reasonable encoding/decoding settings, and provide some error message for other formats if a user attempts to step back, e.g. reverse frame stepping unavailable for current video due to format/encoding/decoding settings.

Yeah, this. That's likely more or less what they already do with the scrub bar.

The back and forth in itself feels so weird to me, with so many hurt feelings:

- the devs expressed in no uncertain terms that they don't want to do it (the first answer is just perfect)

- every third comment is about "we know you don't want to do it, but as users why should we care ?"

Well, if you don't care about the devs, on what base are you asking them to care about your specific problem ?

That blows up pretty quickly though? Even a 10 minute video will take in the worst case ~20 seconds to decode at that rate.

Not really an excuse not to have it (since most video wont be encoded in such an insane way), but the developers owe no obligation to users to implement it.

you'd have to "rebase" all the other frames to be derived from those

> It's of course work to do, and I'm not super motivated to send them that patch, and there's the question of it it would be merged

That's my issue; he calls for people to send patches, but anyone capable of writing such a patch is also probably going to see that he's not positive on the matter, and that his "patches welcome" is really pretty passive aggressive in this instance. At least, that's how it comes off to me. I would expect that, should I submit such a patch, it would simply be rejected on the basis that "it is not a general solution".

> Well, if you don't care about the devs, on what base are you asking them to care about your specific problem ?

Caring about the user's requirements is part of the dev's job description. Caring about the dev's... anything is not in the user's job description. (one advantage commercial software has: it really does help when there's an interface between the dev and the user in the form of customer support. or a commercial incentive to actually work on what the user wants.)

You wouldn't have to any more than you need 0 through N frames in memory to calculate frame N+1. Whatever your decoding state completes at frame N can be considered a key frame.

> Caring about the user's requirements is part of the dev's job description.

For OSS project, it's better to assume that the user persona for the software is the devs or the maintainers. The dev-user relationship you expect is actually the vendor-client in commercial software.

There's also a middle ground: Painstakingly describe the solution first, along with its downside of not being general in the same way as some of the existing features (I guess for example seeking back 10 seconds) are not, and ask whether a patch implementing this solution would be welcome before implementing it.

> job description

Money getting involved would indeed simplify the question.

Here no money is changing hand, so coming up with an angle that's motivating enough for the devs is IMHO the only option. Either bring up an aspect they're not considering that changes the equation for them, or come up with a solution that isn't plaggued by the issues they are afraid to deal with.

That's where I see listening to the devs and caring about their issues to be the only path forward, short of contributing as a dev oneself..

decoding state can be bulkier than a key frame and opaque to the CPU if hardware acceleration is used

I wonder if caching semi-compressed frames would be more efficient in either case (CPU or GPU)

I learned this the hard way.

One other team at my workplace insisted that they can’t make their product compatible with our product, because it would take a team and half year. I knew that it’s a lie, but we convinced them to “allow” us to make for them. I finished - alone - in four days.

It was never merged. It was purely political. It was never about whether it’s possible or not.

Theoretically. Practically, 10+ minute videos with just a single i frame at the beginning do not exist.

all that's correct, but it's besides the point since other players are able to do this.

There is no reason to start caching previous frames until AFTER the user has paused and pressed the "back frame" key. Only THEN does it need to rewind to the previous i-frame and re-render and cache frames. And there is no measurable cost to remembering the timestamp of the last i-frame, so you know where to rewind to.

Oh, I already tried that, and it didn't work.

https://forum.videolan.org/viewtopic.php?f=12&t=103604&p=407...

I wanted to report a big about VLC's extraordinarily badly designed "Magnification/Zoom" user interface, so first I searched the forum to see if there was any other discussion about it, which there naturally was.

So I painstakingly wrote up an extremely detailed description of a bunch of interrelated bugs related to zooming and how it terribly interacted with other features like rotation, in response to the VLC development team brushing off another user complaining about its terrible "Magnification/Zoom" user interface, and they brushed me off too because they were too lazy to read it.

They told me to just submit a bug report, but I pointed out that I was describing a several interrelated bugs, which would require submitting many bug reports, which they would have known if they had actually bothered to read what I painstakingly wrote in great detail with step by step instructions about how to reproduce the bugs and suggestions for improvements, so I obviously wanted to discuss them all first to see if they were even worth my time submitting multiple bug reports about, or if all my efforts reporting bugs and trying to fix them and submit patches would be a waste of time, brushed off and ignored like they did to the other users who described the bugs and usability problems they were experiencing.

Jean-Baptiste Kempf himself replied "If you did shorter posts, maybe people will read them..."

To which I replied "if you did less arrogant responses to long posts, maybe people wouldn't give up on trying to help you."

And of course most of the pathologically terrible bugs I described are still there, a dozen years later. And Jean-Baptiste Kempf still continues to act that way.

More details:

https://news.ycombinator.com/item?id=41281153

HN user KingMob's post perfectly summarized my discouraging experience from a dozen years ago, about a set of bugs and usability problems relating to the horrible "Magnification/Zoom" interface:

https://news.ycombinator.com/item?id=41280375

>KingMob 5 hours ago | unvote | parent | context | flag | favorite | on: Mpv – A free, open-source, and cross-platform medi...

>It's because the developer is misconstruing a non-technical decision they made as a technical limitation. The commenters are trying to point this out, which misses the reality that the developer probably isn't going to budge from their requirement of universal support.

>That dev's rationalization also sends a signal to any commenter with the technical chops to submit a PR, that it will probably be rejected for not supporting 100% of the codecs. I have no doubt people who could do it, over the years looked at that thread and concluded it would be a waste of their time.

Jean-Baptiste Kempf still continues to act that way, and still hasn't even admitted to those bugs and usability problems, let alone fixed them or accepted patches from anyone else who did. He just discourages qualified developers from collaborating, and brushes off legitimate requests from users who can't code but fucking well know other video players don't suffer from those problems.

> I think technically he’s correct (I haven’t worked on media decoding code, but I understand how common video encoding formats work).

He’s technically simply wrong (I have worked on media decoding code, hell I’m working on a related project today). His player supports seeking back by 10 seconds or whatnot, but he insists that somehow to implement precise seeking to the previous frame, you need to seek from the very beginning of the video, no two ways about it:

> There is not a slight technical difficulty. On a logical level, this feature is algorithmically impossible, except for the extreme: You can decode all frames up to the previous one. But this would be far too slow for anything except really short videos.

It’s obvious bullshit, if you encounter one of those pathological videos (that don’t really exist except in his mind or in some test suites) you just give up after a reasonable amount of time, same as how you give up if you can’t seek back ~10s in a reasonable amount of time.

And players do give up seeking all the time already, not just on these hypothetical one-I-frame-per-hour videos, but real world videos with messed up pts/dts with no reasonable way to go back a short interval.

To be fair, as a maintainer I also dread walls of text from super motivated people about details to which I assign very low priority. I’m never an asshole about it, though.

As outlined in some other thread, mpv is not able to stream eg to ChromeCast, unlike VLC. Maybe VLC supports certain things that make the previous-frame thing harder. I suspect it is so, but I don't have insight into the detailed architecture of VLC, unlike I assume the VLC developers. Do you?

Why is their architecture making it harder on them relevant to the question if it's possible? Because the root question is if it's possible, and there's multiple existence proofs that it's possible. Maybe the VLC developers are just tired. I don't blame them. They had to do a whole refactor to get Chromecast support working, and they got no thanks for that. Or maybe it just wasn't enough thanks and they don't feel like doing another refactor. Chromecast support is quite tricky, I've dug into the protocol.

Anyway, I'm not in control of their development, I'm just pointing out that seeking backwards is possible.

To also be fair, to "Painstakingly describe the solution first" absolutely requires a wall of text to enumerate all the multiple layers of interacting bugs, and give step-by-step instructions for reproducing them.

At least I put in the effort to search the discussion group for an existing thread about the problems I had, and contributed to that thread by supporting other users and validating their complaints, instead of opening yet another redundant thread.

The reason I went into so much detail was that the VLC developers were ALREADY acting like assholes by brushing off other people's shorter less detailed descriptions of the same problems, with glib quips like "The holy grail already exists... built in to OS X."

The zooming built into OS X definitely doesn't solve the problems that they refuse to admit exist with their astoundingly terrible "Magnification/Zoom" interface, so I described the problems for their benefit in the same detail I would appreciate in bug reports on my own open source software, in response to their rudely and curtly brushing off other users with the same problems, who don't all have a background in user interface design and software development and writing bug reports.

If the holy grail already exists and solves the problem, then they should REMOVE the horrible unusable "Magnification/Zoom" feature that breaks even worse when you dare to rotate or flip the video, or better yet they should have never allowed that broken "feature" to be merged into VLC in the first place, because of its ridiculously poor design and implementation quality (like drawing and tracking the gui with gigantic fat pixels in un-scaled, un-rotated video pixel coordinates, instead of full resolution screen overlay coordinates, and ignoring the flip/rotation for mouse tracking so you can't see what you're pointing at, which is negligent and insane).

Ironically, VLC accepting and distributing features like the "Magnification/Zoom" interface certainly undermines their arguments that they don't want to accept other patches because of quality and reliability and usability issues. If they refuse to fix it, they should remove it instead, it's just so bad.

And if I didn't bother going to the effort of describing the problems in detail with step-by-step instructions to reproduce them, I'm afraid that Jean-Baptiste Kempf is so thin skinned and arrogant that he would have brushed off my bug report for that reason too. Just like he CONTINUES to rudely and passive-aggressively brush off and ignore other people's perfectly valid bug reports to this day, 12 years later. He's not going to suddenly change.

Or in the less-than-worst case you could cache the created full frame if no native full frame is encountered for X seconds and then in the worst case you don't have to go to the beginning of the video, but to that cached intermediate full frame?

It doesn’t have to be every frame though. Pre calculate to 25%/50%/75%, then as I’m playing, save key frames for more incremental points, and if I start scrubbing, calculate more around that region.

Edit: this doesn’t have to happen synchronously either, it can occur in a background thread or passively.