Yt-dlp: External JavaScript runtime now required for full YouTube support

    yt-dlp --cookies-from-browser firefox --remote-components ejs:github -f "bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best" 'https://www.youtube.com/watch?v=XXX'

    [youtube] [jsc:deno] Solving JS challenges using deno
    [youtube] [jsc:deno] Downloading challenge solver lib script from  https://github.com/yt-dlp/ejs/releases/download/0.3.1/yt.solver.lib.min.js

Download a random video and then copy ejs from yt-dlp’s cache directory (I think it’s in /home/username/.cache)

> being able to package it up together with the solver

`make yt-dlp-extra`

YouTube barely works in a full-on browser these days, props to the team that keeps it accessible via a Python script!

If the concern is security, it sounds like the team went to great lengths to ensure the JS was sandboxed (as long as you’re using Deno).

If you’re using some sort of weird OS or architecture that Deno/Node doesn’t support, you might consider QuickJS, which is written in pure C and should work on anything. (Although it will be a lot slower, I’m not clear just how slow.) Admittedly, you then loose the sandboxing, although IMO it seems like it should safe to trust code being served by Google on the official Youtube domain. (You don’t have to trust Google in general to trust that they won’t serve you actual malware.)

Regardless, Google services getting worse over time is becoming a law rather than a tendency.

On mobile (Firefox) I frequently have issues with videos freezing or videos crashing when I try to replay a section.

I freely admit to holding google software to a higher standard than e.g. random FOSS tools I use or saas from startups, however I also believe google has the talent, time, and money to where their software should basically be the best on Earth, and it's kinda shocking how often it's not and in what ways it's not. And YouTube is how old now?

The fact alone that I still can't toggle off Google maps "we found a faster route, tap ok to not change the route you change" thing...

(This may be because I have a whole swathe of adblockers, etc., plus I do a lot of `yt-dlp`ing from the same IP which may have me on a naughty list.)

They didn't say “can't run JS code”, but that from that location the solver could not be downloaded currently. It could be that it is an IPv6-only environment (IIRC youtube supports IPv6 but github does not), or just that all external sites must be assessed before whitelisted (I'm not sure why youtube would be but not github, but it is certainly possible).

The version in the Arch repos does not include the impersonate feature.

Agreed. Shorts about half the time don't display comments, the back button breaks in mysterious ways. And I use Chrome on both Intel and M macOS machines, so the best in class there is, but my Windows Chrome doesn't fare much better. And Adblock ain't at fault, I pay for premium.

And that's just the technical side. The content side is even worse, comments sections are overrun by bots, not to mention the countless AI slop and content thieves, and for fucks sake I get that high class youtubers have a lot of effort to do to make videos, but why youtube doesn't step in and put clear regulations on sponsorship blocks is beyond me. Betterhelp, AG1, airup, NordVPN (and VPNs in general) should be outright banned.

And the ads, for those who aren't paying for premium, are also just fucked up. Fake game ads (Kingshot who stole sound effects from the original indie Thronefall ...) galore.

Google makes money here, they could go and actually hire a few people to vet ads and police the large youtubers with their sponsors.

What else does youtube music get you? I can play on my phone with the screen off with yt vanced ( and I would never pay just for that feature, because I remember when it was free and they took it away )

Nothing specific, just tend to run tools in restricted VMs where things are whitelisted and it's pretty much as locked down as it can be. It can run whatever I want it to run, including JS, and as the logs in my previous comment shows, it is in fact running both Python and JS, and has access to YouTube, otherwise it wouldn't have worked :)

I tend to have the rule of "least possible privileges" so most stuff I run like that has to be "prepped" basically, especially things that does network requests sometimes (updating the solver in this case), just a matter of packaging it before I run it, so it's not the end of the world.

No weird OS or architecture here, just good ol' Linux.

> IMO it seems like it should safe to trust code being served by Google on the official Youtube domain

The JS script being downloaded is from the yt-dlp GitHub organization (https://github.com/yt-dlp/ejs/releases/download/0.3.1/yt.sol...), not from Google or any websites, FWIW.

But eh either 5s of black screen or 60s of ads. I tried watching a 15 min yt video without adblock and it had 5 ad breaks with some unskippable ads.

I think in this case, my own laziness is what makes it worse than it has to, currently I'm doing whitelisting by domains, so youtube.com for the yt-dlp runner is obviously OK, and I'd want to avoid whitelisting github.com for that, since it's just downloading one JS file.

For now manually copying the config file into my SCM or just whitelisting GitHub for initial download does the trick. I guess I just had to squeeze in one complaint in my previous comment so I could get the HN stamp of approval, can't be too positive.

FWIW, when I use Chromium (logged out/in) on Linux, everything works fine. If I use Firefox (logged in), it works worse. If I change the user-agent to Chromium in Firefox, I get faster buffering than when I use the default user-agent. Make of that what you will.

Around 30-50x slower than V8 (node/deno).

I've been recently benchmarking a lot of different engines: https://ivankra.github.io/javascript-zoo/

With that, uBO and Sponsorblock, I never see any ads and have a great YT experience. (I don't have premium either)

A solver running at 50ms instead of 1ms I would say is practically imperceptible to most users, but I don't know what time span you are measuring with those numbers.

there's a lot of iphone/ipad users out there.

> Everything's already on youtube and they let you make playlists, and they have playlists of the top charts already

I don't use it but ui probably. ads maybe. plenty of people have money and don't want the inconvenience of trying to get around it.

Also, when playing music you won't be hit with ads.

Your setup can move with you wherever you are, home, travel, in the vehicle. This can be helpful for engaging the audible sensors of small aliens sans screen.

Youtube without ads on every device, anywhere, is quite a different experience.

Happy to read and learn more about the setups you've found helpful to do this.

  $ time ./v8 /bench/yt-dlp.js | md5sum -
  a730e32029941bf1f60f9587a6d9554f  -
  real 0m0.252s
  user 0m0.386s
  sys 0m0.074s

  $ time ./quickjs /bench/yt-dlp.js | md5sum -
  a730e32029941bf1f60f9587a6d9554f  -
  real 0m2.280s
  user 0m2.507s
  sys 0m0.031s

A few more results on same input:

  spidermonkey 0.334s
  v8_jitless 1.096s => about the limit for JIT-less interpreters like quickjs
  graaljs 2.396s
  escargot 3.344s
  libjs 4.501s
  brimstone 6.328s
  modernc-quickjs 12.767s (pure Go port of quickjs)
  fastschema-qjs 1m22.801s (Wasm port of quickjs)
  boa 1m28.070s
  quickjs-ng 2m49.202s

Something has really changed to the worse lately. I think it has to do with anti-ad programs as well as AI, like the UI also changed. It is important to point out that while you do not have any issues, other people do or may.

No. Because even if it might be complicated, any website developer can test their website against a wide array of browsers, in a more or less automated way.

Actually I think this is kind of a yt-dlp bug, since it doesn't need to use IPv6 for the github download.

No, Firefox always been a second-rate guest at Google properties, and I'm not expecting it to change soon either. Why would they make it better when status quo means more Chrome users (in their mind)?

    WARNING: [youtube] [jsc] Remote components challenge solver script (deno) and NPM package (deno) were skipped. These may be required to solve JS challenges. You can enable these downloads with  --remote-components ejs:github  (recommended) or  --remote-components ejs:npm , respectively. For more information and alternatives, refer to  https://github.com/yt-dlp/yt-dlp/wiki/EJS

Edit: Maybe I misunderstood, now when I re-read your post. You meant it'll prevent the automatic download at runtime perhaps? That sounds about right if so.

I would expect YouTube managers to pressure the Chrome managers, because YouTube brings in billions of dollars every month. Likewise I would expect the trend to move in favor of YouTube, because the browser loses money at an increasing amount and YouTube generates money at an increasing amount. 70% of YouTube happens on Mobile, and in the US more people are now watching on TVs than phones. Source: Nielsen, the old-school company that has huge influence over ads.

The site pops a literal warning saying "having problems? turn off your ad blocker" so I'm not sure where the mysteries lie here.

They're testing on thousands of devices. And they're probably even testing against ad-blockers on your bro-browser. But they're certainly not motivated to optimize that experience, so you get what you get.

I meant the challenge that is the reason they need the Javascript in the first place.

You can’t very well run yt-dlp without trusting yt-dlp code.

Not only do I get slowdowns and some videos don't load at all at times, but I also get a notification that explains that the reason is using adblockers.

Also many times, the video won't play. If I reload the page, then it will play.

I tried it on my slower laptop. I get:

   node(v8)  : 1.25s user 0.12s system 154% cpu 0.892 total
   quickjs   : 6.54s user 0.11s system 99% cpu 6.671 total
   quickjs-ng: 545.55s user 202.67s system 99% cpu 12:32.28 total

For example I have to scroll down a lot to get to the comment section, the suggestions are all over the place, and so forth. Annoying.

Also due to uBlock Origin, some videos do not start and I have to refresh. It is not much of an issue for me but the fact that apparently I need a huge monitor to see the "old layout" is a problem for me.

It feels like something the FTC should be investigating, or perhaps a European equivalent, but I doubt it will.

[1]: https://news.ycombinator.com/item?id=41379517

> > IMO it seems like it should safe to trust code being served by Google on the official Youtube domain

Which came from a misunderstanding about where the downloadable solver script comes from, as it doesn't come from youtube.com, it comes from github.com (yt-dlp org), I was just correcting that misunderstanding.

> You can’t very well run yt-dlp without trusting yt-dlp code.

That makes a ton of sense and I agree! I'm not sure how that is related to anything though? I download yt-dlp from Arch repositories, so yes I'm trusting Arch maintainers and of course yt-dlp developers. Then I'm adding a manifest which controls what this application can actually access, which is basically a VM config, where I define that it can access youtube.com (and a bunch of other sites I mirror/archive). This is the part that shouldn't have github.com/* access.

Again as mentioned, not a big issue, plenty of workarounds, so not the end of the world.

When Firefox dies, the last glimmer of hope will go out.

Discord has started to become absurdly aggressive with it too, to the point that they don't even let you load messages whilst logged in if you're on a VPN.

It really makes me feel like there will be an inflection point in a few years, where the internet is cleaved in two. You'll have the 'free' internet that is full of interesting stuff but also full of malware, spam and scams, and you'll have the squeaky clean corporate internet, basically a facsimile of WeChat's super-app, of which you'll only have access with a government ID. No VPNs or anti-fingerprinting allowed.

...Can they not just bundle a solver? For that matter, deno is available as a PyPI package.

> as I'm probably not alone in running yt-dlp in a restricted environment

...An environment that doesn't allow you to install Deno, but does allow you to install yt-dlp?

Notably, YouTube these days prioritize AV1 codec even if you don’t have gpu acceleration for it, making lots of systems fall back to CPU decoding and making it completely unusable. Install the h264ify extension to force h264 during content negotiation and get your gpu decoding back.

Even if you can make a matrix of all those combinations, it’s even more complex than that to test in practice. Take my laptop for example, it starts off good and manages the cpu decoding for a while, a few minutes into a video it overheats and throttles, causing stutter.

What YouTube should do on the other hand, and I’m sure they already do, is to collect metrics from all playbacks. That should show black on white how many users struggle with each codec.

I don’t think I’m in any minority here given how many million installations the h264ify extension has. Google simply care more about their bandwidth cost than the user experience.

There a bunch of bad actors doing mass scrape of all public server and history via all of those VPN

But that script is ultimately running a JS challenge from Youtube, right? That’s why we actually needed a JS runtime in the first place.

I wonder if being a YouTube Premium subscriber is also a factor here. I do pay for it so I don't see ads. But maybe the way ads are being served/injected has changed things for the worse for people that get them.

    FROM python:3-slim
    RUN python3 -m pip --no-cache-dir install 'yt-dlp[default]'
    RUN apt-get update \
     && DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -q -y ffmpeg curl unzip \
     && curl -fsSL https://deno.land/install.sh -o /tmp/deno.sh \
     && sh /tmp/deno.sh -y \
     && mv /root/.deno/bin/deno /usr/local/bin/ \
     && rm --force --recursive /var/lib/apt/lists/* /tmp/* /var/tmp/*
    ENTRYPOINT ["yt-dlp"]

Yeah - I watch most of my YouTubes on the Apple TV and the ads are a pestilence. Sometimes it'll be 50s pre-roll[1] with multiple 30-50s breaks for a 10m videos.

Luckily there exist[0] many fine technologies that let you view them without ads via something like Infuse with a DLNA server if you're that way inclined.

[0] Currently. YT-DLP is fighting the good fight but I don't know how much longer they'll be able to keep in front. But then I'll just stop watching YouTube, really, because it's a horror show without adblock/circumventions.

[1] The video doesn't appear in your history until the pre-roll has finished which means if you can't be arsed sitting through a 50s pre-roll just that second and - at least on the Apple TV - you've not clicked on the video from your homepage / subscriptions, good luck trying to find it again unless you remember the name + channel etc. (which it also won't properly show you until after the pre-roll!)[2]

[2] I hate YouTube corporate.

See https://www.reddit.com/r/youtube/comments/1id4amh/youtube_ch... and https://discussions.apple.com/thread/255931836

Hasn't it been like that for already many years?

arguably, already 90's AOL very much pushed its users to stay within its walled garden.