Discussion: Handling YouTube's new required preroll waiting period

[alexeichhorn]

YouTube recently internally introduced something that sometimes prevents streams from being accessed instantly. Instead, one has to wait a couple of seconds - simulating the ad preroll, I guess.

This has been previously reported in yt-dlp in yt-dlp/yt-dlp#13930 and been "solved" in this commit yt-dlp/yt-dlp@a97f4cb.
It was solved by actually waiting that amount of time during download.

Now, YouTubeKit doesn't have any downloading functionality, and this can still be an issue here, where streams sometimes return 403 errors.

This issue is mostly just to bring awareness. I will think about how to potentially help handle this. Any ideas are welcome ❤️

[alexanderyuryev]

Yes, same issue since August 23. On average 20-25% of the time the link returns 403.
And it seems that the share is growing every day.
Is this issue only with livestreams or regular direct links have the same behavior?
@alexeichhorn How hard is it to port this yt-dlp/yt-dlp@a97f4cb solution to YouTubeKit (I'm not quite sure what you mean about waiting while downloading.) ?

[RufusMall]

Interesting.

Does this mean that after we have called something like: streams.highestAudioBitrateStream()

We should sleep for 6 seconds before downloading from the stream.url?

[mdfalcon104]

Today this issue occurred. With the same URL, it wasn’t working on the latest version, but after waiting a few hours and trying again, it’s working now. Could YouTube be blocking something internally?

[alexeichhorn]

Interesting.

Does this mean that after we have called something like: streams.highestAudioBitrateStream()

We should sleep for 6 seconds before downloading from the stream.url?

That's how yt-dlp implements it. But since YouTubeKit doesn't handle downloads internally, it's harder to force sleep for certain sources, I guess. I could just sleep in general before returning any source, but this seems not like a super good solution.

[alexanderyuryev]

Interesting.
Does this mean that after we have called something like: streams.highestAudioBitrateStream()
We should sleep for 6 seconds before downloading from the stream.url?

That's how yt-dlp implements it. But since YouTubeKit doesn't handle downloads internally, it's harder to force sleep for certain sources, I guess. I could just sleep in general before returning any source, but this seems not like a super good solution.

You mean the link you get returns 403 for the first 6 seconds and 200 after that?
I checked. 6 seconds, 30 seconds. Nothing happens.
BTW, the same link works differently on different networks (50/50 on Wi-Fi, 100% works on LTE).
I guess YouTube is just blocking some of the traffic (maybe too many requests or something).

[alexeichhorn]

Yeah, don't know exactly what's the full logic behind it.
But importantly, the link will return 403 forever, when accessed within the x seconds of the waiting period. So, you can't just spam it until it works. It will never work, if you do this. That's at least my testing experience.

[alexanderyuryev]

Yeah, don't know exactly what's the full logic behind it. But importantly, the link will return 403 forever, when accessed within the x seconds of the waiting period. So, you can't just spam it until it works. It will never work, if you do this. That's at least my testing experience.

As far as I understand, YoutubeKit does not call the link inside the library?
So you can wait until the first call, and it should work.
Correct me if I'm wrong.

In practice it does not work.

If you request livestreams, the link (.m3u8 file) works and returns a playlist text file. Inside the file links to the playlists of different formats and resolutions (.m3u8 file again) and they are also available and already inside them the links to the media files (.ts files) return 403

I am not sure that problem is preroll waiting period

[chnbr]

I am running 0.2.9 but the problem still persists on certain machines.

During testing I have experienced that on two of my Macs it is working fine and on my M4 Mac I get 403 response (same Video, same executable). My implementation is probably OK since it exists for more than a year now and worked well until now.

I have also made tests with delaying the download and can confirm it does NOT help to wait 6+ seconds.

Further: When I run YTK on my M4 Mac it does fail on every video (in Safari they play OK). When I run YTK with the same app compiled for iPhone (which is connected to the very same Mac and sharing Internet connection, so the IP is identical) it does work.

How can YT distinguish between iPhone and Mac when everything else (IP, executable) are identical ?

I hope these findings might help somehow. Any ideas what else to test ?

[RufusMall]

You

I am running 0.2.9 but the problem still persists on certain machines.

During testing I have experienced that on two of my Macs it is working fine and on my M4 Mac I get 403 response (same Video, same executable). My implementation is probably OK since it exists for more than a year now and worked well until now.

I have also made tests with delaying the download and can confirm it does NOT help to wait 6+ seconds.

Further: When I run YTK on my M4 Mac it does fail on every video (in Safari they play OK). When I run YTK with the same app compiled for iPhone (which is connected to the very same Mac and sharing Internet connection, so the IP is identical) it does work.

How can YT distinguish between iPhone and Mac when everything else (IP, executable) are identical ?

I hope these findings might help somehow. Any ideas what else to test ?

You could maybe try putting a tool like "Proxyman" in-between and see what the difference is in the requests?

[chnbr]

You could maybe try putting a tool like "Proxyman" in-between and see what the difference is in the requests?

Well, yes, but the requests look similar regarding some parameters. YT however includes some sort of one-time signature and expiration that are different all the time (even when originated from the same source). So, the gain in information would be nearly NULL. The download link of the resource is different each request. You can compare much.

[hugosilvac]

Hi everyone 👋

I’m facing the same issue mentioned here, where the audioURL returns 403 (access denied). I’ve tested in different scenarios, but so far I haven’t found any way to bypass this limitation.

Has anyone managed to find an alternative, adjustment, or workaround that is currently working?
Any information would be really helpful. 🙏

Thanks!

[Josscii]

youtube seems to block the web client streams

[alexeichhorn]

Maybe to fix this, we could handle it similar to this: yt-dlp/yt-dlp#14646