`videos?part=short` does not work sometimes when the video is just uploaded

[Benjamin-Loison]

Tracked at Benjamin_Loison/youtube_push_notifications/issues/33.

[Benjamin-Loison]

Related to #26 and #330.

[Benjamin-Loison]

YouTube video: bvG2xqIvdLg (source: https://matrix.to/#/!sNARMdEsFZERaQAJzl:matrix.org/$hCXGPPKpl96BEC099MRKDvTnyPJzuF_3LP0w58leG8w) is a short, not confuse with YouTube video: yjHWvfwfBxU (source: https://matrix.to/#/!sNARMdEsFZERaQAJzl:matrix.org/$fXXhUeNKA0DY3Nl_AIpSD1ln8mNJY6zi-0bfv-RUSu0).

log.txt:

...
dataStr: {
    "kind": "youtube#videoListResponse",
    "etag": "NotImplemented",
    "items": [
        {
            "kind": "youtube#video",
            "etag": "NotImplemented",
            "id": "bvG2xqIvdLg",
            "short": {
                "available": false
            },
            "captions": []
        }
    ]
}|
short available: |
...

On OverClock3000:

curl -k 'https://youtube.local/videos?part=short&id=bvG2xqIvdLg'

Output:

{
    "kind": "youtube#videoListResponse",
    "etag": "NotImplemented",
    "items": [
        {
            "kind": "youtube#video",
            "etag": "NotImplemented",
            "id": "bvG2xqIvdLg",
            "short": {
                "available": true
            }
        }
    ]
}

[Benjamin-Loison]

YouTube-operational-API/videos.php

Line 174 in 0d2768a

'available' => !isRedirection("https://www.youtube.com/shorts/$id")

[Benjamin-Loison]

YouTube-operational-API/common.php

Lines 88 to 102 in 0d2768a

	function isRedirection($url)
	{
	$opts = [
	'http' => [
	'ignore_errors' => true,
	'follow_location' => false,
	]
	];
	$http_response_header = getHeadersFromOpts($url, $opts);
	$code = intval(explode(' ', $http_response_header[0])[1]);
	if (in_array($code, HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC)) {
	detectedAsSendingUnusualTraffic();
	}
	return $code == 303;
	}

[Benjamin-Loison]

grep -rn 'follow_location'

common.php:102:                'follow_location' => false,

[Benjamin-Loison]

curl -k 'https://youtube.local/videos?part=short&id=bvG2xqIvdLg'

Output:

{
    "kind": "youtube#videoListResponse",
    "etag": "NotImplemented",
    "items": [
        {
            "kind": "youtube#video",
            "etag": "NotImplemented",
            "id": "bvG2xqIvdLg",
            "short": {
                "http_response_header": {
                    "0": "HTTP\/1.1 200 OK",
                    "Content-Type": "text\/html; charset=utf-8",
                    "X-Content-Type-Options": "nosniff",
                    "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate",
                    "Pragma": "no-cache",
                    "Expires": "Mon, 01 Jan 1990 00:00:00 GMT",
                    "Date": "Sat, 13 Sep 2025 12:34:06 GMT",
                    "Strict-Transport-Security": "max-age=31536000",
                    "X-Frame-Options": "SAMEORIGIN",
                    "Content-Security-Policy": "require-trusted-types-for 'script'",
                    "Report-To": "{\"group\":\"youtube_main\",\"max_age\":2592000,\"endpoints\":[{\"url\":\"https:\/\/csp.withgoogle.com\/csp\/report-to\/youtube_main\"}]}",
                    ...
                    "Alt-Svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000",
                    "Accept-Ranges": "none",
                    "Vary": "Accept-Encoding",
                    "Connection": "close",
                    "Transfer-Encoding": "chunked"
                },
                "available": true
            }
        }
    ]
}

[Benjamin-Loison]

diff:

diff --git a/common.php b/common.php
index ac3a4ce..9100827 100644
--- a/common.php
+++ b/common.php
@@ -85,7 +85,16 @@
         return [$result, $http_response_header];
     }
 
-    function isRedirection($url)
+    function isRedirectionFromHeaders($http_response_header)
+    {
+        $code = intval(explode(' ', $http_response_header[0])[1]);
+        if (in_array($code, HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC)) {
+            detectedAsSendingUnusualTraffic();
+        }
+        return $code == 303;
+    }
+
+    function getRedirectionHeaders($url)
     {
         $opts = [
             'http' => [
@@ -94,11 +103,13 @@
             ]
         ];
         $http_response_header = getHeadersFromOpts($url, $opts);
-        $code = intval(explode(' ', $http_response_header[0])[1]);
-        if (in_array($code, HTTP_CODES_DETECTED_AS_SENDING_UNUSUAL_TRAFFIC)) {
-            detectedAsSendingUnusualTraffic();
-        }
-        return $code == 303;
+        return $http_response_header;
+    }
+
+    function isRedirection($url)
+    {
+        $http_response_header = getRedirectionHeaders($url);
+        return isRedirectionFromHeaders($http_response_header);
     }
 
     function getRemote($url, $opts = [], $verifyTrafficIfForbidden = true)
diff --git a/videos.php b/videos.php
index cd98016..85c8866 100644
--- a/videos.php
+++ b/videos.php
@@ -170,8 +170,10 @@
         }
 
         if ($options['short']) {
+            $http_response_header = getRedirectionHeaders("https://www.youtube.com/shorts/$id");
             $short = [
-                'available' => !isRedirection("https://www.youtube.com/shorts/$id")
+                'http_response_header' => $http_response_header,
+                'available' => !isRedirectionFromHeaders($http_response_header),//isRedirection("https://www.youtube.com/shorts/$id")
             ];
             $item['short'] = $short;
         }

[Benjamin-Loison]

So now let us wait the issue to happen again.

[Benjamin-Loison]

log.txt:

...
<details>
<summary><code>log.txt</code>:</summary>

...
dataStr: {
"kind": "youtube#videoListResponse",
"etag": "NotImplemented",
"items": [
{
"kind": "youtube#video",
"etag": "NotImplemented",
"id": "Gdqpg6VYF3k",
"short": {
"http_response_header": {
"0": "HTTP/1.1 303 See Other",
"Content-Type": "application/binary",
"X-Content-Type-Options": "nosniff",
"Cache-Control": "no-cache, no-store, max-age=0, must-revalidate",
"Pragma": "no-cache",
"Expires": "Mon, 01 Jan 1990 00:00:00 GMT",
"Date": "Sat, 13 Sep 2025 19:47:16 GMT",
"Location": "https://www.youtube.com/watch?v=Gdqpg6VYF3k",
"X-Frame-Options": "SAMEORIGIN",
"Strict-Transport-Security": "max-age=31536000",
"Permissions-Policy": "ch-ua-arch=, ch-ua-bitness=, ch-ua-full-version=, ch-ua-full-version-list=, ch-ua-model=, ch-ua-wow64=, ch-ua-form-factors=, ch-ua-platform=, ch-ua-platform-version=*",
"Cross-Origin-Opener-Policy": "same-origin-allow-popups; report-to="youtube_main"",
"Content-Security-Policy": "require-trusted-types-for 'script'",
...
"Alt-Svc": "h3=":443"; ma=2592000,h3-29=":443"; ma=2592000",
"Connection": "close"
},
"available": false
},
"captions": [
{
"name": "French (auto-generated)",
"languageCode": "fr",
"kind": "asr"
}
]
}
]
}
...

</details>
...
dataStr: {
    "kind": "youtube#videoListResponse",
    "etag": "NotImplemented",
    "items": [
        {
            "kind": "youtube#video",
            "etag": "NotImplemented",
            "id": "RtzAKgJZ3Rc",
            "short": {
                "http_response_header": {
                    "0": "HTTP\/1.1 303 See Other",
                    "Content-Type": "application\/binary",
                    "X-Content-Type-Options": "nosniff",
                    "Cache-Control": "no-cache, no-store, max-age=0, must-revalidate",
                    "Pragma": "no-cache",
                    "Expires": "Mon, 01 Jan 1990 00:00:00 GMT",
                    "Date": "Sat, 13 Sep 2025 18:49:27 GMT",
                    "Location": "https:\/\/www.youtube.com\/watch?v=RtzAKgJZ3Rc",
                    "X-Frame-Options": "SAMEORIGIN",
                    "Strict-Transport-Security": "max-age=31536000",
                    "Content-Security-Policy": "require-trusted-types-for 'script'",
                    "Cross-Origin-Opener-Policy": "same-origin-allow-popups; report-to=\"youtube_main\"",
                    "Permissions-Policy": "ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-form-factors=*, ch-ua-platform=*, ch-ua-platform-version=*",
                    "Report-To": "{\"group\":\"youtube_main\",\"max_age\":2592000,\"endpoints\":[{\"url\":\"https:\/\/csp.withgoogle.com\/csp\/report-to\/youtube_main\"}]}",
                    ...
                    "Alt-Svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000",
                    "Connection": "close"
                },
                "available": false
            },
            "captions": []
        }
    ]
}
...

So both are 303.

[Benjamin-Loison]

The Stack Overflow answers 75476455 and 76602819 seem the most promising.

Related to #159.

[Benjamin-Loison]

DuckDuckGo and Google search youtubeAnalytics Python.

https://developers.google.com/youtube/reporting/v1/code_samples/python
https://developers.google.com/youtube/analytics/dimensions#channel

Output:

{
  "error": {
    "code": 403,
    "message": "Forbidden",
    "errors": [
      {
        "message": "Forbidden",
        "domain": "global",
        "reason": "forbidden"
      }
    ]
  }
}

[Benjamin-Loison]

Let us retrieve playlist items before whether the video is a short, that way if the latter does not work, while the former does, then there is room for improvement on my end.

https://www.youtube.com/channel/UCt5USYpzzMCYhkirVQGHwKQ
http://localhost/YouTube-operational-API/noKey/playlistItems?part=snippet&playlistId=UUSHt5USYpzzMCYhkirVQGHwKQ&fields=items/snippet/resourceId/videoId&maxResults=1

Output:

{
  "error": {
    "code": 404,
    "message": "The playlist identified with the request's \u003ccode\u003eplaylistId\u003c/code\u003e parameter cannot be found.",
    "errors": [
      {
        "message": "The playlist identified with the request's \u003ccode\u003eplaylistId\u003c/code\u003e parameter cannot be found.",
        "domain": "youtube.playlistItem",
        "reason": "playlistNotFound",
        "location": "playlistId",
        "locationType": "parameter"
      }
    ]
  }
}