[ie/apple] Rework extractors (#17277)

* Add API fallback for ApplePodcastsIE
* Fix AppleConnectIE token extraction
* Refactor and add AppleBaseIE

Closes #17266
Authored by: bashonly

.github/workflows/release.yml

@@ -280,7 +280,7 @@ jobs:
           fi
           printf '\n\n%s\n\n%s%s%s\n\n---\n' \
             "#### A description of the various files is in the [README](https://github.com/${REPOSITORY}#release-files)" \
-            "The zipimport Unix executable contains code licensed under ISC and MIT. " \
+            "The zipimport Unix executable and release tarball contain code licensed under ISC and MIT. " \
             "The PyInstaller-bundled executables are subject to these and other licenses, all of which are compiled in " \
             "[THIRD_PARTY_LICENSES.txt](https://github.com/${BASE_REPO}/blob/${HEAD_SHA}/THIRD_PARTY_LICENSES.txt)" >> ./RELEASE_NOTES
           python ./devscripts/make_changelog.py -vv --collapsible >> ./RELEASE_NOTES

README.md

@@ -142,11 +142,11 @@ While yt-dlp is licensed under the [Unlicense](LICENSE), many of the release fil
 
 Most notably, the PyInstaller-bundled executables include GPLv3+ licensed code, and as such the combined work is licensed under [GPLv3+](https://www.gnu.org/licenses/gpl-3.0.html).
 
-The zipimport Unix executable (`yt-dlp`) contains [ISC](https://github.com/meriyah/meriyah/blob/main/LICENSE.md) licensed code from [`meriyah`](https://github.com/meriyah/meriyah) and [MIT](https://github.com/davidbonnet/astring/blob/main/LICENSE) licensed code from [`astring`](https://github.com/davidbonnet/astring).
+The zipimport Unix executable (`yt-dlp`) and release tarball (`yt-dlp.tar.gz`) contain [ISC](https://github.com/meriyah/meriyah/blob/main/LICENSE.md) licensed code from [`meriyah`](https://github.com/meriyah/meriyah) and [MIT](https://github.com/davidbonnet/astring/blob/main/LICENSE) licensed code from [`astring`](https://github.com/davidbonnet/astring).
 
 See [THIRD_PARTY_LICENSES.txt](THIRD_PARTY_LICENSES.txt) for more details.
 
-The git repository, the source tarball (`yt-dlp.tar.gz`), the PyPI source distribution and the PyPI built distribution (wheel) only contain code licensed under the [Unlicense](LICENSE).
+The git repository, the PyPI source distribution and the PyPI built distribution (wheel) only contain code licensed under the [Unlicense](LICENSE).
 
 <!-- MANPAGE: END EXCLUDED SECTION -->
 
@@ -1859,7 +1859,7 @@ The following extractors use this feature:
 #### youtube
 * `lang`: Prefer translated metadata (`title`, `description` etc) of this language code (case-sensitive). By default, the video primary language metadata is preferred, with a fallback to `en` translated. See [youtube/_base.py](https://github.com/yt-dlp/yt-dlp/blob/415b4c9f955b1a0391204bd24a7132590e7b3bdb/yt_dlp/extractor/youtube/_base.py#L402-L409) for the list of supported content language codes
 * `skip`: One or more of `hls`, `dash` or `translated_subs` to skip extraction of the m3u8 manifests, dash manifests and [auto-translated subtitles](https://github.com/yt-dlp/yt-dlp/issues/4090#issuecomment-1158102032) respectively
-* `player_client`: Clients to extract video data from. The currently available clients are `web`, `web_safari`, `web_embedded`, `web_music`, `web_creator`, `mweb`, `ios`, `android`, `android_vr`, `tv`, `tv_downgraded`, and `tv_simply`. By default, `android_vr,web_safari` is used. If no JavaScript runtime/engine is available, then only `android_vr` is used. If logged-in cookies are passed to yt-dlp, then `tv_downgraded,web_safari` is used for free accounts and `tv_downgraded,web_creator` is used for premium accounts. The `web_music` client is added for `music.youtube.com` URLs when logged-in cookies are used. The `web_embedded` client is added for age-restricted videos but only successfully works around the age-restriction sometimes (e.g. if the video is embeddable), and may be added as a fallback if `android_vr` is unable to access a video. The `web_creator` client is added for age-restricted videos if account age-verification is required. Some clients, such as `web_creator` and `web_music`, require a `po_token` for their formats to be downloadable. Some clients, such as `web_creator`, will only work with authentication. Not all clients support authentication via cookies. You can use `default` for the default clients, or you can use `all` for all clients (not recommended). You can prefix a client with `-` to exclude it, e.g. `youtube:player_client=default,-web_safari`
+* `player_client`: Clients to extract video data from. The currently available clients are `web`, `web_safari`, `web_embedded`, `web_music`, `web_creator`, `mweb`, `ios`, `visionos`, `android`, `android_vr`, `tv`, `tv_downgraded`, and `tv_simply`. By default, `visionos,android_vr,web` is used. If no JavaScript runtime/engine is available, then `web` is omitted. If logged-in cookies are passed to yt-dlp, then `tv_downgraded,web` is used for free accounts and `tv_downgraded,web_creator,web` is used for premium accounts. The `web_music` client is added for `music.youtube.com` URLs when logged-in cookies are used. The `web_embedded` client is added for age-restricted videos but only successfully works around the age-restriction sometimes (e.g. if the video is embeddable). The `tv_downgraded` client may be added as a fallback if `android_vr` or `visionos` is unable to access a video. The `web_creator` client is added for age-restricted videos if account age-verification is required. Some clients, such as `web_creator` and `web_music`, require a `po_token` for their formats to be downloadable. Some clients, such as `web_creator`, will only work with authentication. Not all clients support authentication via cookies. You can use `default` for the default clients, or you can use `all` for all clients (not recommended). You can prefix a client with `-` to exclude it, e.g. `youtube:player_client=default,-web`
 * `player_skip`: Skip some network requests that are generally needed for robust extraction. One or more of `configs` (skip client configs), `webpage` (skip initial webpage), `js` (skip js player), `initial_data` (skip initial data/next ep request). While these options can help reduce the number of requests needed or avoid some rate-limiting, they could cause issues such as missing formats or metadata.  See [#860](https://github.com/yt-dlp/yt-dlp/pull/860) and [#12826](https://github.com/yt-dlp/yt-dlp/issues/12826) for more details
 * `webpage_skip`: Skip extraction of embedded webpage data. One or both of `player_response`, `initial_data`. These options are for testing purposes and don't skip any network requests. Neither is skipped by default; however, if a `player_js_version` value other than `actual` is used, then `webpage_skip=player_response` is implied
 * `webpage_client`: Client to use for the video webpage request. One of `web` or `web_safari` (default)

bundle/pyinstaller.py

@@ -38,11 +38,16 @@ def main():
         f'--name={name}',
         '--icon=devscripts/logo.ico',
         '--upx-exclude=vcruntime140.dll',
+        # setuptools and packaging are PyInstaller runtime dependencies,
+        # but would be collected due to cffi's imports if we don't exclude
+        '--exclude-module=setuptools',
+        '--exclude-module=packaging',
         # Ref: https://github.com/yt-dlp/yt-dlp/issues/13311
         #      https://github.com/pyinstaller/pyinstaller/issues/9149
         '--exclude-module=pkg_resources',
         '--noconfirm',
         '--additional-hooks-dir=yt_dlp/__pyinstaller',
+        '--add-data=THIRD_PARTY_LICENSES.txt:.',
         *opts,
         'yt_dlp/__main__.py',
     ]

devscripts/generate_third_party_licenses.py

@@ -10,7 +10,7 @@ HEADER = '''THIRD-PARTY LICENSES
 This file aggregates license texts of third-party components included with the yt-dlp PyInstaller-bundled executables.
 yt-dlp itself is licensed under the Unlicense (see LICENSE file).
 Source code for bundled third-party components is available from the original projects.
-If you cannot obtain it, the maintainers will provide it as per license obligation; maintainer emails are listed in pyproject.toml.'''
+If you cannot obtain it, the maintainers will provide it as per license obligation: email maintainers@yt-dlp.org'''
 
 
 @dataclass(frozen=True)
@@ -47,6 +47,13 @@ DEPENDENCIES: list[Dependency] = [
         license_url='https://raw.githubusercontent.com/libffi/libffi/refs/heads/master/LICENSE',
         project_url='https://sourceware.org/libffi/',
     ),
+    Dependency(
+        name='OpenSSL 1.x',
+        license='OpenSSL',
+        license_url='https://raw.githubusercontent.com/openssl/openssl/refs/tags/OpenSSL_1_1_1t/LICENSE',
+        comment='Only included in `yt-dlp.exe` and `yt-dlp_x86.exe` Windows builds',
+        project_url='https://www.openssl.org/',
+    ),
     Dependency(
         name='OpenSSL 3.0+',
         license='Apache-2.0',
@@ -123,43 +130,79 @@ DEPENDENCIES: list[Dependency] = [
         name='libintl',
         license='LGPL-2.1-or-later',
         license_url='https://raw.githubusercontent.com/autotools-mirror/gettext/refs/heads/master/gettext-runtime/intl/COPYING.LIB',
-        comment='Only included in macOS builds',
+        comment='Only included in Linux builds',
         project_url='https://www.gnu.org/software/gettext/',
     ),
     Dependency(
         name='libidn2',
         license='LGPL-3.0-or-later',
         license_url='https://gitlab.com/libidn/libidn2/-/raw/master/COPYING.LESSERv3',
-        comment='Only included in macOS builds',
+        comment='Only included in Linux builds',
         project_url='https://www.gnu.org/software/libidn/',
     ),
     Dependency(
         name='libidn2 (Unicode character data files)',
         license='Unicode-TOU AND Unicode-DFS-2016',
         license_url='https://gitlab.com/libidn/libidn2/-/raw/master/COPYING.unicode',
-        comment='Only included in macOS builds',
+        comment='Only included in Linux builds',
         project_url='https://www.gnu.org/software/libidn/',
     ),
     Dependency(
         name='libunistring',
         license='LGPL-3.0-or-later',
         license_url='https://gitweb.git.savannah.gnu.org/gitweb/?p=libunistring.git;a=blob_plain;f=COPYING.LIB;hb=HEAD',
-        comment='Only included in macOS builds',
+        comment='Only included in Linux builds',
         project_url='https://www.gnu.org/software/libunistring/',
     ),
+
+    # Non-Python dependencies of curl_cffi
     Dependency(
-        name='librtmp',
+        name='curl-impersonate',
-        license='LGPL-2.1-or-later',
+        license='MIT',
-        # No official repo URL
+        license_url='https://raw.githubusercontent.com/lexiforest/curl-impersonate/refs/heads/main/LICENSE',
-        license_url='https://gist.githubusercontent.com/seproDev/31d8c691ccddebe37b8b379307cb232d/raw/053408e98547ea8c7d9ba3a80c965f33e163b881/librtmp_COPYING.txt',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
-        comment='Only included in macOS builds',
+        project_url='https://github.com/lexiforest/curl-impersonate',
-        project_url='https://rtmpdump.mplayerhq.hu/',
+    ),
+    Dependency(
+        name='curl',
+        license='curl',
+        license_url='https://raw.githubusercontent.com/curl/curl/refs/heads/master/LICENSES/curl.txt',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://curl.se/',
+    ),
+    Dependency(
+        name='BoringSSL',
+        license='Apache-2.0',
+        license_url='https://raw.githubusercontent.com/google/boringssl/refs/heads/main/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://boringssl.googlesource.com/boringssl',
+    ),
+    Dependency(
+        name='nghttp2',
+        license='MIT',
+        license_url='https://raw.githubusercontent.com/nghttp2/nghttp2/refs/heads/master/COPYING',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://nghttp2.org/',
+    ),
+    Dependency(
+        name='ngtcp2',
+        license='MIT',
+        license_url='https://raw.githubusercontent.com/ngtcp2/ngtcp2/refs/heads/main/COPYING',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://nghttp2.org/ngtcp2/',
+    ),
+    Dependency(
+        name='nghttp3',
+        license='MIT',
+        license_url='https://raw.githubusercontent.com/ngtcp2/nghttp3/refs/heads/main/COPYING',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://nghttp2.org/nghttp3/',
     ),
     Dependency(
         name='zstd',
         license='BSD-3-Clause',
         license_url='https://raw.githubusercontent.com/facebook/zstd/refs/heads/dev/LICENSE',
-        comment='Only included in macOS builds',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
         project_url='https://facebook.github.io/zstd/',
     ),
 
@@ -174,28 +217,74 @@ DEPENDENCIES: list[Dependency] = [
         name='curl_cffi',
         license='MIT',
         license_url='https://raw.githubusercontent.com/lexiforest/curl_cffi/refs/heads/main/LICENSE',
-        comment='Not included in `yt-dlp_x86` and `yt-dlp_musllinux_aarch64` builds',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
         project_url='https://curl-cffi.readthedocs.io/',
     ),
+    # curl_cffi vendored code:
+    # - https://github.com/lexiforest/curl_cffi/blob/v0.15.0/curl_cffi/_asyncio_selector.py
+    Dependency(
+        name='Tornado',
+        license='Apache-2.0',
+        license_url='https://raw.githubusercontent.com/tornadoweb/tornado/master/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='http://www.tornadoweb.org/',
+    ),
+    # curl_cffi vendored code:
+    # - https://github.com/lexiforest/curl_cffi/blob/v0.15.0/curl_cffi/requests/cookies.py
+    # - https://github.com/lexiforest/curl_cffi/blob/v0.15.0/curl_cffi/requests/headers.py
+    Dependency(
+        name='httpx',
+        license='BSD-3-Clause',
+        license_url='https://github.com/encode/httpx/raw/master/LICENSE.md',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://www.python-httpx.org/',
+    ),
     # Dependency of curl_cffi
     Dependency(
-        name='curl-impersonate',
+        name='rich',
         license='MIT',
-        license_url='https://raw.githubusercontent.com/lexiforest/curl-impersonate/refs/heads/main/LICENSE',
+        license_url='https://raw.githubusercontent.com/Textualize/rich/refs/heads/main/LICENSE',
-        comment='Not included in `yt-dlp_x86` and `yt-dlp_musllinux_aarch64` builds',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
-        project_url='https://github.com/lexiforest/curl-impersonate',
+        project_url='https://rich.readthedocs.io/',
     ),
+    # Dependency of rich
+    Dependency(
+        name='pygments',
+        license='BSD-2-Clause',
+        license_url='https://raw.githubusercontent.com/pygments/pygments/refs/heads/master/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='http://pygments.org/',
+    ),
+    # Dependency of rich
+    Dependency(
+        name='markdown-it-py',
+        license='MIT',
+        license_url='https://raw.githubusercontent.com/executablebooks/markdown-it-py/refs/heads/master/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://markdown-it-py.readthedocs.io/',
+    ),
+    # Dependency of markdown-it-py
+    Dependency(
+        name='mdurl',
+        license='MIT',
+        license_url='https://raw.githubusercontent.com/executablebooks/mdurl/refs/heads/master/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
+        project_url='https://github.com/executablebooks/mdurl',
+    ),
+    # Dependency of cryptography and curl_cffi
     Dependency(
         name='cffi',
         license='MIT-0',  # Technically does not need to be included
         license_url='https://raw.githubusercontent.com/python-cffi/cffi/refs/heads/main/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
         project_url='https://cffi.readthedocs.io/',
     ),
-    # Dependecy of cffi
+    # Dependency of cffi
     Dependency(
         name='pycparser',
         license='BSD-3-Clause',
         license_url='https://raw.githubusercontent.com/eliben/pycparser/refs/heads/main/LICENSE',
+        comment='Not included in `yt-dlp_x86.exe` Windows builds',
         project_url='https://github.com/eliben/pycparser',
     ),
     Dependency(
@@ -276,12 +365,14 @@ DEPENDENCIES: list[Dependency] = [
         name='Meriyah',
         license='ISC',
         license_url='https://raw.githubusercontent.com/meriyah/meriyah/refs/heads/main/LICENSE.md',
+        comment='Also included in `yt-dlp` zipimport Unix executables and `yt-dlp.tar.gz` release tarballs',
         project_url='https://github.com/meriyah/meriyah',
     ),
     Dependency(
         name='Astring',
         license='MIT',
         license_url='https://raw.githubusercontent.com/davidbonnet/astring/refs/heads/main/LICENSE',
+        comment='Also included in `yt-dlp` zipimport Unix executables and `yt-dlp.tar.gz` release tarballs',
         project_url='https://github.com/davidbonnet/astring/',
     ),
 ]

yt_dlp/downloader/hls.py

@@ -300,7 +300,7 @@ class HlsFD(FragmentFD):
 
         # We only download the first fragment during the test
         if self.params.get('test', False):
-            fragments = [fragments[0] if fragments else None]
+            fragments = fragments[:1]
 
         if real_downloader:
             info_dict['fragments'] = fragments

yt_dlp/extractor/_extractors.py

@@ -106,7 +106,6 @@ from .archiveorg import (
 )
 from .arcpublishing import ArcPublishingIE
 from .ard import (
-    ARDIE,
     ARDAudiothekIE,
     ARDAudiothekPlaylistIE,
     ARDBetaMediathekIE,
@@ -1719,7 +1718,10 @@ from .shahid import (
 from .sharepoint import SharePointIE
 from .shemaroome import ShemarooMeIE
 from .shiey import ShieyIE
-from .showroomlive import ShowRoomLiveIE
+from .showroomlive import (
+    ShowRoomLiveIE,
+    ShowRoomVodIE,
+)
 from .sibnet import SibnetEmbedIE
 from .simplecast import (
     SimplecastEpisodeIE,

yt_dlp/extractor/americastestkitchen.py

@@ -1,12 +1,16 @@
-import json
+import itertools
+import re
 
 from .common import InfoExtractor
+from ..networking.exceptions import HTTPError
 from ..utils import (
+    ExtractorError,
     clean_html,
     int_or_none,
     try_get,
     unified_strdate,
     unified_timestamp,
+    urljoin,
 )
 
 
@@ -17,7 +21,7 @@ class AmericasTestKitchenIE(InfoExtractor):
         'md5': 'b861c3e365ac38ad319cfd509c30577f',
         'info_dict': {
             'id': '5b400b9ee338f922cb06450c',
-            'title': 'Japanese Suppers',
+            'title': 'Weeknight Japanese Suppers',
             'ext': 'mp4',
             'display_id': 'weeknight-japanese-suppers',
             'description': 'md5:64e606bfee910627efc4b5f050de92b3',
@@ -106,23 +110,44 @@ class AmericasTestKitchenIE(InfoExtractor):
 
 
 class AmericasTestKitchenSeasonIE(InfoExtractor):
-    _VALID_URL = r'https?://(?:www\.)?(?P<show>americastestkitchen|(?P<cooks>cooks(?:country|illustrated)))\.com(?:(?:/(?P<show2>cooks(?:country|illustrated)))?(?:/?$|(?<!ated)(?<!ated\.com)/episodes/browse/season_(?P<season>\d+)))'
+    _VALID_URL = r'''(?x)
+        https?://(?:www\.)?(?P<domain>americastestkitchen|cookscountry|cooksillustrated)\.com
+        (?:/(?P<path>cookscountry|cooksillustrated))?
+        (?:/episodes(?:/browse)?/season[-_](?P<season>\d+))?
+        /?(?:[?#]|$)'''
+    _SHOWS = {
+        'americastestkitchen': ('', 'America\'s Test Kitchen'),
+        'cookscountry': ('/cookscountry', 'Cook\'s Country'),
+        'cooksillustrated': ('/cooksillustrated', 'Cook\'s Illustrated'),
+    }
     _TESTS = [{
         # ATK Season
-        'url': 'https://www.americastestkitchen.com/episodes/browse/season_1',
+        'url': 'https://www.americastestkitchen.com/episodes/season-1',
         'info_dict': {
             'id': 'season_1',
             'title': 'Season 1',
         },
-        'playlist_count': 13,
+        'playlist_mincount': 13,
+    }, {
+        # Latest ATK Season (new URL scheme)
+        'url': 'https://www.americastestkitchen.com/episodes/season-26',
+        'info_dict': {
+            'id': 'season_26',
+            'title': 'Season 26',
+        },
+        'playlist_count': 26,
     }, {
         # Cooks Country Season
-        'url': 'https://www.americastestkitchen.com/cookscountry/episodes/browse/season_12',
+        'url': 'https://www.americastestkitchen.com/cookscountry/episodes/season-12',
         'info_dict': {
             'id': 'season_12',
             'title': 'Season 12',
         },
-        'playlist_count': 13,
+        'playlist_mincount': 13,
+    }, {
+        # Old-style URL (redirects to the new season page)
+        'url': 'https://www.americastestkitchen.com/episodes/browse/season_1',
+        'only_matching': True,
     }, {
         # America's Test Kitchen Series
         'url': 'https://www.americastestkitchen.com/',
@@ -130,7 +155,7 @@ class AmericasTestKitchenSeasonIE(InfoExtractor):
             'id': 'americastestkitchen',
             'title': 'America\'s Test Kitchen',
         },
-        'playlist_count': 558,
+        'playlist_mincount': 558,
     }, {
         # Cooks Country Series
         'url': 'https://www.americastestkitchen.com/cookscountry',
@@ -138,7 +163,7 @@ class AmericasTestKitchenSeasonIE(InfoExtractor):
             'id': 'cookscountry',
             'title': 'Cook\'s Country',
         },
-        'playlist_count': 199,
+        'playlist_mincount': 199,
     }, {
         'url': 'https://www.americastestkitchen.com/cookscountry/',
         'only_matching': True,
@@ -157,59 +182,46 @@ class AmericasTestKitchenSeasonIE(InfoExtractor):
     }]
 
     def _real_extract(self, url):
-        season_number, show1, show = self._match_valid_url(url).group('season', 'show', 'show2')
+        domain, url_path, season = self._match_valid_url(url).group('domain', 'path', 'season')
-        show_path = ('/' + show) if show else ''
+        show_path, title = self._SHOWS[url_path or domain]
-        show = show or show1
+        season = int_or_none(season)
-        season_number = int_or_none(season_number)
 
-        slug, title = {
+        if season:
-            'americastestkitchen': ('atk', 'America\'s Test Kitchen'),
+            playlist_id = f'season_{season}'
-            'cookscountry': ('cco', 'Cook\'s Country'),
+            playlist_title = f'Season {season}'
-            'cooksillustrated': ('cio', 'Cook\'s Illustrated'),
-        }[show]
 
-        facet_filters = [
+            def entries():
-            'search_document_klass:episode',
+                yield from self._season_entries(show_path, season)
-            'search_show_slug:' + slug,
-        ]
-
-        if season_number:
-            playlist_id = f'season_{season_number}'
-            playlist_title = f'Season {season_number}'
-            facet_filters.append('search_season_list:' + playlist_title)
         else:
-            playlist_id = show
+            playlist_id = url_path or domain
             playlist_title = title
 
-        season_search = self._download_json(
+            def entries():
-            f'https://y1fnzxui30-dsn.algolia.net/1/indexes/everest_search_{slug}_season_desc_production',
+                for season_number in itertools.count(1):
-            playlist_id, headers={
+                    try:
-                'Origin': 'https://www.americastestkitchen.com',
+                        yield from self._season_entries(show_path, season_number)
-                'X-Algolia-API-Key': '8d504d0099ed27c1b73708d22871d805',
+                    except ExtractorError as e:
-                'X-Algolia-Application-Id': 'Y1FNZXUI30',
+                        if isinstance(e.cause, HTTPError) and e.cause.status == 404:
-            }, query={
+                            break
-                'facetFilters': json.dumps(facet_filters),
+                        raise
-                'attributesToRetrieve': f'description,search_{slug}_episode_number,search_document_date,search_url,title,search_atk_episode_season',
-                'attributesToHighlight': '',
-                'hitsPerPage': 1000,
-            })
-
-        def entries():
-            for episode in (season_search.get('hits') or []):
-                search_url = episode.get('search_url')  # always formatted like '/episode/123-title-of-episode'
-                if not search_url:
-                    continue
-                yield {
-                    '_type': 'url',
-                    'url': f'https://www.americastestkitchen.com{show_path or ""}{search_url}',
-                    'id': try_get(episode, lambda e: e['objectID'].split('_')[-1]),
-                    'title': episode.get('title'),
-                    'description': episode.get('description'),
-                    'timestamp': unified_timestamp(episode.get('search_document_date')),
-                    'season_number': season_number,
-                    'episode_number': int_or_none(episode.get(f'search_{slug}_episode_number')),
-                    'ie_key': AmericasTestKitchenIE.ie_key(),
-                }
 
         return self.playlist_result(
             entries(), playlist_id, playlist_title)
+
+    def _season_entries(self, show_path, season_number):
+        webpage = self._download_webpage(
+            f'https://www.americastestkitchen.com{show_path}/episodes/season-{season_number}',
+            f'season-{season_number}', f'Downloading season {season_number} webpage')
+        seen = set()
+        for episode in re.finditer(
+                r'<a [^>]*\bhref="(?P<path>/(?:cookscountry/|cooksillustrated/)?episode/(?P<id>\d+)-[^"]+)"[^>]*>\s*<h3[^>]*>(?P<title>[^<]+)</h3>',
+                webpage):
+            path = episode.group('path')
+            if path in seen:
+                continue
+            seen.add(path)
+            yield self.url_result(
+                urljoin('https://www.americastestkitchen.com', path),
+                AmericasTestKitchenIE, episode.group('id'),
+                clean_html(episode.group('title')),
+                season_number=season_number)

yt_dlp/extractor/appleconnect.py

@@ -1,39 +1,22 @@
-import time
+from .applepodcasts import AppleBaseIE
-
-from .common import InfoExtractor
 from ..utils import (
-    ExtractorError,
-    extract_attributes,
     float_or_none,
-    jwt_decode_hs256,
-    jwt_encode,
     parse_resolution,
     qualities,
     unified_strdate,
     update_url,
     url_or_none,
-    urljoin,
 )
 from ..utils.traversal import (
-    find_element,
     require,
     traverse_obj,
 )
 
 
-class AppleConnectIE(InfoExtractor):
+class AppleConnectIE(AppleBaseIE):
     IE_NAME = 'apple:music:connect'
     IE_DESC = 'Apple Music Connect'
 
-    _BASE_URL = 'https://music.apple.com'
-    _QUALITIES = {
-        'provisionalUploadVideo': None,
-        'sdVideo': 480,
-        'sdVideoWithPlusAudio': 480,
-        'sd480pVideo': 480,
-        '720pHdVideo': 720,
-        '1080pHdVideo': 1080,
-    }
     _VALID_URL = r'https?://music\.apple\.com/[\w-]+/post/(?P<id>\d+)'
     _TESTS = [{
         'url': 'https://music.apple.com/us/post/1018290019',
@@ -59,29 +42,16 @@ class AppleConnectIE(InfoExtractor):
         },
     }]
 
-    _jwt = None
+    _BASE_URL = 'https://music.apple.com'
-
+    _JWT_KEY_ID = 'WebPlayKid'
-    @staticmethod
+    _QUALITIES = {
-    def _jwt_is_expired(token):
+        'provisionalUploadVideo': None,
-        return jwt_decode_hs256(token)['exp'] - time.time() < 120
+        'sdVideo': 480,
-
+        'sdVideoWithPlusAudio': 480,
-    def _get_token(self, webpage, video_id):
+        'sd480pVideo': 480,
-        if self._jwt and not self._jwt_is_expired(self._jwt):
+        '720pHdVideo': 720,
-            return self._jwt
+        '1080pHdVideo': 1080,
-
+    }
-        js_url = traverse_obj(webpage, (
-            {find_element(tag='script', attr='crossorigin', value='', html=True)},
-            {extract_attributes}, 'src', {urljoin(self._BASE_URL)}, {require('JS URL')}))
-        js = self._download_webpage(
-            js_url, video_id, 'Downloading token JS', 'Unable to download token JS')
-
-        header = jwt_encode({}, '', headers={'alg': 'ES256', 'kid': 'WebPlayKid'}).split('.')[0]
-        self._jwt = self._search_regex(
-            fr'(["\'])(?P<jwt>{header}(?:\.[\w-]+){{2}})\1', js, 'JSON Web Token', group='jwt')
-        if self._jwt_is_expired(self._jwt):
-            raise ExtractorError('The fetched token is already expired')
-
-        return self._jwt
 
     def _real_extract(self, url):
         video_id = self._match_id(url)

yt_dlp/extractor/applepodcasts.py

@@ -1,15 +1,55 @@
+import time
+
 from .common import InfoExtractor
 from ..utils import (
+    ExtractorError,
     clean_html,
     clean_podcast_url,
     int_or_none,
+    jwt_decode_hs256,
+    jwt_encode,
     parse_iso8601,
+    try_call,
+    update_url,
+    url_or_none,
+    urljoin,
 )
 from ..utils.traversal import traverse_obj
 
 
-class ApplePodcastsIE(InfoExtractor):
+class AppleBaseIE(InfoExtractor):
-    _VALID_URL = r'https?://podcasts\.apple\.com/(?:[^/]+/)?podcast(?:/[^/]+){1,2}.*?\bi=(?P<id>\d+)'
+    """Subclasses must set _BASE_URL and _JWT_KEY_ID"""
+
+    _jwt_cache = {}
+
+    @staticmethod
+    def _jwt_is_expired(token):
+        return jwt_decode_hs256(token)['exp'] - time.time() < 120
+
+    def _get_token(self, webpage, episode_id):
+        if self._jwt_cache.get(self._BASE_URL) and not self._jwt_is_expired(self._jwt_cache[self._BASE_URL]):
+            return self._jwt
+
+        js_path = self._search_regex(
+            r'<script [^>]*\bsrc="(/assets/index~[0-9a-f]+\.js)">', webpage, 'JS asset path')
+        js_code = self._download_webpage(
+            urljoin(self._BASE_URL, js_path), episode_id,
+            'Downloading JS asset', 'Unable to download JS asset')
+
+        header = jwt_encode({}, '', headers={'typ': 'JWT', 'alg': 'ES256', 'kid': self._JWT_KEY_ID}).split('.')[0]
+        self._jwt_cache[self._BASE_URL] = self._search_regex(
+            fr'(["\'])(?P<jwt>{header}(?:\.[\w-]+){{2}})\1', js_code, 'JSON Web Token', group='jwt')
+        if self._jwt_is_expired(self._jwt_cache[self._BASE_URL]):
+            raise ExtractorError('The fetched token is already expired')
+
+        return self._jwt_cache[self._BASE_URL]
+
+
+class ApplePodcastsIE(AppleBaseIE):
+    IE_NAME = 'apple:podcasts'
+    IE_DESC = 'Apple Podcasts'
+
+    _VALID_URL = r'https?://podcasts\.apple\.com/(?P<country>[^/?#]+/)?podcast(?:/[^/?#]+){1,2}/?\?(?:[^#]+&)?i=(?P<id>\d+)'
     _TESTS = [{
         'url': 'https://podcasts.apple.com/us/podcast/urbana-podcast-724-by-david-penn/id1531349107?i=1000748574256',
         'md5': 'f8a6f92735d0cfbd5e6a7294151e28d8',
@@ -23,7 +63,7 @@ class ApplePodcastsIE(InfoExtractor):
             'timestamp': 1770400801,
             'duration': 3602,
             'series': 'Urbana Radio Show',
-            'thumbnail': 're:.+[.](png|jpe?g|webp)',
+            'thumbnail': r're:https://.+/.+\.jpg',
         },
     }, {
         'url': 'https://podcasts.apple.com/us/podcast/207-whitney-webb-returns/id1135137367?i=1000482637777',
@@ -39,7 +79,7 @@ class ApplePodcastsIE(InfoExtractor):
             'timestamp': 1593932400,
             'duration': 5369,
             'series': 'The Tim Dillon Show',
-            'thumbnail': 're:.+[.](png|jpe?g|webp)',
+            'thumbnail': r're:https://.+/.+\.jpg',
         },
     }, {
         'url': 'https://podcasts.apple.com/podcast/207-whitney-webb-returns/id1135137367?i=1000482637777',
@@ -52,15 +92,55 @@ class ApplePodcastsIE(InfoExtractor):
         'only_matching': True,
     }]
 
-    def _real_extract(self, url):
+    _BASE_URL = 'https://podcasts.apple.com'
-        episode_id = self._match_id(url)
+    _JWT_KEY_ID = 'C4J7GBP74H'
-        webpage = self._download_webpage(url, episode_id)
+
+    def _extract_podcast_from_api(self, webpage, episode_id, country_code):
+        data = self._download_json(
+            f'https://amp-api.podcasts.apple.com/v1/catalog/{country_code or "us"}/podcast-episodes/{episode_id}',
+            episode_id, headers={
+                'Authorization': f'Bearer {self._get_token(webpage, episode_id)}',
+                'Origin': self._BASE_URL,
+            },
+            query={
+                # XXX: if video is available, try adding the params 'with=entitlements,hlsVideo'
+                'extend': 'fullDescription',
+                'include': 'podcast',
+                'l': 'en-US',
+            })['data'][0]
+
+        thumb_info = traverse_obj(data, ('attributes', 'artwork', {
+            'url': ('url', {url_or_none}),
+            'h': ('height', {int_or_none}),
+            'w': ('width', {int_or_none}),
+        }))
+
+        return {
+            'id': episode_id,
+            **traverse_obj(data, {
+                'title': ('attributes', 'name', {str}),
+                'description': ('attributes', 'fullDescription', {clean_html}),
+                'url': ('attributes', 'assetUrl', {clean_podcast_url}, {update_url(scheme='https')}),
+                'timestamp': ('attributes', 'releaseDateTime', {parse_iso8601}),
+                'duration': ('attributes', 'durationInMilliseconds', {int_or_none(scale=1000)}),
+                'episode': ('attributes', 'name', {str}),
+                'episode_number': ('attributes', 'episodeNumber', {int_or_none}),
+                'series': ('relationships', 'podcast', 'data', 0, 'attributes', 'name', {str}),
+            }),
+            'thumbnail': try_call(lambda: thumb_info.pop('url').format(f='jpg', **thumb_info)),
+            'vcodec': 'none',
+        }
+
+    def _extract_podcast_from_webpage(self, webpage, episode_id):
         server_data = self._search_json(
             r'<script [^>]*\bid=["\']serialized-server-data["\'][^>]*>', webpage,
-            'server data', episode_id)['data'][0]['data']
+            'server data', episode_id, default=None)
         model_data = traverse_obj(server_data, (
-            'headerButtonItems', lambda _, v: v['$kind'] == 'share' and v['modelType'] == 'EpisodeLockup',
+            'data', 0, 'data', 'headerButtonItems',
+            lambda _, v: v['$kind'] == 'share' and v['modelType'] == 'EpisodeLockup',
             'model', {dict}, any))
+        if not model_data:
+            return None
 
         return {
             'id': episode_id,
@@ -77,3 +157,12 @@ class ApplePodcastsIE(InfoExtractor):
             'thumbnail': self._og_search_thumbnail(webpage),
             'vcodec': 'none',
         }
+
+    def _real_extract(self, url):
+        episode_id, country_code = self._match_valid_url(url).group('id', 'country')
+        # Webpage may be unavailable, see https://github.com/yt-dlp/yt-dlp/issues/17266
+        webpage = self._download_webpage(url, episode_id, expected_status=500)
+
+        return (
+            self._extract_podcast_from_webpage(webpage, episode_id)
+            or self._extract_podcast_from_api(webpage, episode_id, country_code))

yt_dlp/extractor/ard.py

@@ -11,15 +11,12 @@ from ..utils import (
     join_nonempty,
     jwt_decode_hs256,
     make_archive_id,
-    parse_duration,
     parse_iso8601,
     remove_start,
     str_or_none,
-    unified_strdate,
     update_url,
     update_url_query,
     url_or_none,
-    xpath_text,
 )
 from ..utils.traversal import traverse_obj, value
 
@@ -119,118 +116,6 @@ class ARDMediathekBaseIE(InfoExtractor):
         return formats
 
 
-class ARDIE(InfoExtractor):
-    _VALID_URL = r'(?P<mainurl>https?://(?:www\.)?daserste\.de/(?:[^/?#&]+/)+(?P<id>[^/?#&]+))\.html'
-    _TESTS = [{
-        # available till 7.12.2023
-        'url': 'https://www.daserste.de/information/talk/maischberger/videos/maischberger-video-424.html',
-        'md5': '94812e6438488fb923c361a44469614b',
-        'info_dict': {
-            'id': 'maischberger-video-424',
-            'display_id': 'maischberger-video-424',
-            'ext': 'mp4',
-            'duration': 4452.0,
-            'title': 'maischberger am 07.12.2022',
-            'upload_date': '20221207',
-            'thumbnail': r're:^https?://.*\.jpg$',
-        },
-    }, {
-        'url': 'https://www.daserste.de/information/politik-weltgeschehen/morgenmagazin/videosextern/dominik-kahun-aus-der-nhl-direkt-zur-weltmeisterschaft-100.html',
-        'only_matching': True,
-    }, {
-        'url': 'https://www.daserste.de/information/nachrichten-wetter/tagesthemen/videosextern/tagesthemen-17736.html',
-        'only_matching': True,
-    }, {
-        'url': 'https://www.daserste.de/unterhaltung/serie/in-aller-freundschaft-die-jungen-aerzte/videos/diversity-tag-sanam-afrashteh100.html',
-        'only_matching': True,
-    }, {
-        'url': 'http://www.daserste.de/information/reportage-dokumentation/dokus/videos/die-story-im-ersten-mission-unter-falscher-flagge-100.html',
-        'only_matching': True,
-    }, {
-        'url': 'https://www.daserste.de/unterhaltung/serie/in-aller-freundschaft-die-jungen-aerzte/Drehpause-100.html',
-        'only_matching': True,
-    }, {
-        'url': 'https://www.daserste.de/unterhaltung/film/filmmittwoch-im-ersten/videos/making-ofwendezeit-video-100.html',
-        'only_matching': True,
-    }]
-
-    def _real_extract(self, url):
-        mobj = self._match_valid_url(url)
-        display_id = mobj.group('id')
-
-        player_url = mobj.group('mainurl') + '~playerXml.xml'
-        doc = self._download_xml(player_url, display_id)
-        video_node = doc.find('./video')
-        upload_date = unified_strdate(xpath_text(
-            video_node, './broadcastDate'))
-        thumbnail = xpath_text(video_node, './/teaserImage//variant/url')
-
-        formats = []
-        for a in video_node.findall('.//asset'):
-            file_name = xpath_text(a, './fileName', default=None)
-            if not file_name:
-                continue
-            format_type = a.attrib.get('type')
-            format_url = url_or_none(file_name)
-            if format_url:
-                ext = determine_ext(file_name)
-                if ext == 'm3u8':
-                    formats.extend(self._extract_m3u8_formats(
-                        format_url, display_id, 'mp4', entry_protocol='m3u8_native',
-                        m3u8_id=format_type or 'hls', fatal=False))
-                    continue
-                elif ext == 'f4m':
-                    formats.extend(self._extract_f4m_formats(
-                        update_url_query(format_url, {'hdcore': '3.7.0'}),
-                        display_id, f4m_id=format_type or 'hds', fatal=False))
-                    continue
-            f = {
-                'format_id': format_type,
-                'width': int_or_none(xpath_text(a, './frameWidth')),
-                'height': int_or_none(xpath_text(a, './frameHeight')),
-                'vbr': int_or_none(xpath_text(a, './bitrateVideo')),
-                'abr': int_or_none(xpath_text(a, './bitrateAudio')),
-                'vcodec': xpath_text(a, './codecVideo'),
-                'tbr': int_or_none(xpath_text(a, './totalBitrate')),
-            }
-            server_prefix = xpath_text(a, './serverPrefix', default=None)
-            if server_prefix:
-                f.update({
-                    'url': server_prefix,
-                    'playpath': file_name,
-                })
-            else:
-                if not format_url:
-                    continue
-                f['url'] = format_url
-            formats.append(f)
-
-        _SUB_FORMATS = (
-            ('./dataTimedText', 'ttml'),
-            ('./dataTimedTextNoOffset', 'ttml'),
-            ('./dataTimedTextVtt', 'vtt'),
-        )
-
-        subtitles = {}
-        for subsel, subext in _SUB_FORMATS:
-            for node in video_node.findall(subsel):
-                subtitles.setdefault('de', []).append({
-                    'url': node.attrib['url'],
-                    'ext': subext,
-                })
-
-        return {
-            'id': xpath_text(video_node, './videoId', default=display_id),
-            'formats': formats,
-            'subtitles': subtitles,
-            'display_id': display_id,
-            'title': video_node.find('./title').text,
-            'duration': parse_duration(video_node.find('./duration').text),
-            'upload_date': upload_date,
-            'thumbnail': thumbnail,
-        }
-
-
 class ARDBetaMediathekIE(InfoExtractor):
     IE_NAME = 'ARDMediathek'
     _VALID_URL = r'''(?x)https?://

yt_dlp/extractor/instagram.py

@@ -42,7 +42,6 @@ def _id_to_pk(shortcode):
 
 
 class InstagramBaseIE(InfoExtractor):
-    _API_BASE_URL = 'https://i.instagram.com/api/v1'
     _BASE_URL = 'https://www.instagram.com/'
     _APP_IDS = {
         'ios': '124024574287414',
@@ -75,6 +74,12 @@ class InstagramBaseIE(InfoExtractor):
     def _is_web_app(self):
         return self._app_id == self._APP_IDS['web']
 
+    @property
+    def _API_BASE_URL(self):
+        if not self._is_web_app:
+            return 'https://i.instagram.com/api/v1'
+        return 'https://www.instagram.com/api/v1'
+
     @property
     def _api_headers(self):
         return {
@@ -87,7 +92,8 @@ class InstagramBaseIE(InfoExtractor):
 
     @staticmethod
     def _is_login_redirect(url):
-        return urllib.parse.urlparse(url).path.startswith('/accounts/login')
+        path = urllib.parse.urlparse(url).path
+        return path.startswith('/accounts/login') or path == '/'
 
     def _get_count(self, media, kind, *keys):
         return traverse_obj(

yt_dlp/extractor/showroomlive.py

@@ -1,80 +1,134 @@
+import datetime as dt
+import json
+
 from .common import InfoExtractor
 from ..utils import (
-    ExtractorError,
+    UserNotLive,
+    clean_html,
+    extract_attributes,
     int_or_none,
-    urljoin,
+    str_or_none,
+    url_or_none,
+)
+from ..utils.traversal import (
+    find_element,
+    require,
+    traverse_obj,
 )
 
 
 class ShowRoomLiveIE(InfoExtractor):
-    _WORKING = False
+    IE_NAME = 'showroom:live'
-    _VALID_URL = r'https?://(?:www\.)?showroom-live\.com/(?!onlive|timetable|event|campaign|news|ranking|room)(?P<id>[^/?#&]+)'
+    IE_DESC = 'SHOWROOM'
-    _TEST = {
+
-        'url': 'https://www.showroom-live.com/48_Nana_Okada',
+    _VALID_URL = r'https?://(?:www\.)?showroom-live\.com/r/(?P<id>[\w-]+)'
+    _TESTS = [{
+        'url': 'https://www.showroom-live.com/r/48_Yui_Oguri',
         'only_matching': True,
-    }
+    }]
 
     def _real_extract(self, url):
         broadcaster_id = self._match_id(url)
+        webpage = self._download_webpage(
+            url, broadcaster_id, headers={'Accept-Language': 'ja'})
+        nuxt_data = self._search_nuxt_json(webpage, broadcaster_id)['data']
 
-        webpage = self._download_webpage(url, broadcaster_id)
+        cookies = self._get_cookies(url)
+        sr_id = traverse_obj(cookies, ('sr_id', 'value', {str}, filter))
+        if not sr_id:
+            self.raise_login_required()
 
-        room_id = self._search_regex(
+        room_profile = traverse_obj(nuxt_data, (
-            (r'SrGlobal\.roomId\s*=\s*(\d+)',
+            f'roomProfile-{broadcaster_id}-{sr_id}', {dict}))
-             r'(?:profile|room)\?room_id\=(\d+)'), webpage, 'room_id')
+        start_timestamp = traverse_obj(room_profile, ('current_live_started_at', {int_or_none}))
+        is_live = traverse_obj(room_profile, ('is_onlive', {bool}))
 
-        room = self._download_json(
+        if not is_live:
-            urljoin(url, f'/api/room/profile?room_id={room_id}'),
+            if start_timestamp:
-            broadcaster_id)
+                start_time = dt.datetime.fromtimestamp(
+                    start_timestamp, dt.timezone.utc,
+                ).astimezone().strftime('%Y-%m-%d %H:%M:%S %Z')
 
-        is_live = room.get('is_onlive')
+                self.raise_no_formats(
-        if is_live is not True:
+                    f'Next livestream is scheduled to start at {start_time}', expected=True)
-            raise ExtractorError(f'{broadcaster_id} is offline', expected=True)
 
-        uploader = room.get('performer_name') or broadcaster_id
+                return {
-        title = room.get('room_name') or room.get('main_name') or uploader
+                    'id': broadcaster_id,
+                    'live_status': 'is_upcoming',
+                    'release_timestamp': start_timestamp,
+                }
+            raise UserNotLive(video_id=broadcaster_id)
+
+        room_id = traverse_obj(room_profile, ('room_id', {str_or_none}))
+        room_name = traverse_obj(room_profile, (
+            ('room_name', 'main_name'), {clean_html}, filter, any))
 
         streaming_url_list = self._download_json(
-            urljoin(url, f'/api/live/streaming_url?room_id={room_id}'),
+            'https://www.showroom-live.com/api/live/streaming_url',
-            broadcaster_id)['streaming_url_list']
+            broadcaster_id, query={'room_id': room_id})
-
+        m3u8_url = traverse_obj(streaming_url_list, (
-        formats = []
+            'streaming_url_list', lambda _, v: v['type'] == 'hls_all',
-        for stream in streaming_url_list:
+            'url', {url_or_none}, any, {require('m3u8 URL')}))
-            stream_url = stream.get('url')
-            if not stream_url:
-                continue
-            stream_type = stream.get('type')
-            if stream_type == 'hls':
-                m3u8_formats = self._extract_m3u8_formats(
-                    stream_url, broadcaster_id, ext='mp4', m3u8_id='hls',
-                    live=True)
-                for f in m3u8_formats:
-                    f['quality'] = int_or_none(stream.get('quality', 100))
-                formats.extend(m3u8_formats)
-            elif stream_type == 'rtmp':
-                stream_name = stream.get('stream_name')
-                if not stream_name:
-                    continue
-                formats.append({
-                    'url': stream_url,
-                    'play_path': stream_name,
-                    'page_url': url,
-                    'player_url': 'https://www.showroom-live.com/assets/swf/v3/ShowRoomLive.swf',
-                    'rtmp_live': True,
-                    'ext': 'flv',
-                    'format_id': 'rtmp',
-                    'format_note': stream.get('label'),
-                    'quality': int_or_none(stream.get('quality', 100)),
-                })
 
         return {
-            'id': str(room.get('live_id') or broadcaster_id),
+            'title': room_name,
-            'title': title,
+            'channel': room_name,
-            'description': room.get('description'),
+            'channel_id': broadcaster_id,
-            'timestamp': int_or_none(room.get('current_live_started_at')),
+            'formats': self._extract_m3u8_formats(m3u8_url, broadcaster_id, 'mp4'),
-            'uploader': uploader,
+            'is_live': is_live,
-            'uploader_id': broadcaster_id,
+            'release_timestamp': start_timestamp,
-            'view_count': int_or_none(room.get('view_num')),
+            **traverse_obj(room_profile, {
-            'formats': formats,
+                'id': ('live_id', {str_or_none}),
-            'is_live': True,
+                'channel_follower_count': ('follower_num', {int_or_none}),
+                'channel_is_verified': ('is_official', {bool}),
+                'description': ('description', {clean_html}, filter),
+                'genres': ('genre_name', {clean_html}, filter, all, filter),
+                'tags': ('live_tags', ..., 'name', {clean_html}, filter, all, filter),
+                'thumbnail': ('image_square', {url_or_none}),
+                'view_count': ('view_num', {int_or_none}),
+            }),
+        }
+
+
+class ShowRoomVodIE(InfoExtractor):
+    IE_NAME = 'showroom:vod'
+
+    _VALID_URL = r'https?://(?:www\.)?showroom-live\.com/episode/watch\?(?:[^#]+&)?id=(?P<id>\w+)'
+    _TESTS = [{
+        'url': 'https://www.showroom-live.com/episode/watch?id=214',
+        'info_dict': {
+            'id': '214',
+            'ext': 'mp4',
+            'title': 'aaa',
+        },
+    }]
+
+    def _real_extract(self, url):
+        episode_id = self._match_id(url)
+        webpage = self._download_webpage(url, episode_id)
+        episode_data = traverse_obj(webpage, (
+            {find_element(id='episode-data', html=True)},
+            {extract_attributes}, 'data-episode', {json.loads}, {dict}))
+
+        streaming_url_list = self._download_json(
+            'https://www.showroom-live.com/api/episode/streaming_url',
+            episode_id, query={'episode_id': episode_id})
+        m3u8_url = traverse_obj(streaming_url_list, (
+            'streaming_url_list', 'hls_all',
+            'hls_all', {url_or_none}, {require('m3u8 URL')}))
+
+        return {
+            'id': episode_id,
+            'formats': self._extract_m3u8_formats(m3u8_url, episode_id, 'mp4'),
+            'thumbnail': traverse_obj(streaming_url_list, ('thumbnail_url', {url_or_none})),
+            **traverse_obj(episode_data, {
+                'title': ('title', {clean_html}, filter),
+                'description': ('description', {clean_html}, filter),
+                'duration': ('video_time', {int_or_none}),
+                'timestamp': ('display_started_at', {int_or_none}),
+            }),
+            **traverse_obj(episode_data, ('series', {
+                'series': ('name', {clean_html}, filter),
+                'series_id': ('id', {str_or_none}),
+            })),
         }

yt_dlp/extractor/vimeo.py

@@ -49,7 +49,7 @@ class VimeoBaseInfoExtractor(InfoExtractor):
         'Cannot download embed-only video without embedding URL. Please call yt-dlp '
         'with the URL of the page that embeds this video.')
 
-    _DEFAULT_CLIENT = 'macos'
+    _DEFAULT_CLIENT = 'web'
     _DEFAULT_AUTHED_CLIENT = 'web'
     _CLIENT_HEADERS = {
         'Accept': 'application/vnd.vimeo.*+json; version=3.4.10',
@@ -89,7 +89,7 @@ class VimeoBaseInfoExtractor(InfoExtractor):
         },
         'macos': {
             'CACHE_KEY': 'oauth-token-macos',
-            'CACHE_ONLY': False,
+            'CACHE_ONLY': True,
             'VIEWER_JWT': False,
             'REQUIRES_AUTH': False,
             'AUTH': 'NDc1N2JlN2Y5ZjZmMjU3NzE3NTRkZTg1NmY2YzU2MTI0OTFlNjJiYjpwVUNDWUlBZmZqSHhQcndBYWxGMzgyYys2NkN5d1JrREJZZXdPcEdsU05tdjFlVVo2aE1lYk9GcWE3ZW9KVldlYnFlOWh5Vno5UWtpUGJ5empYZFBpYkFwV0FFTnB5VWV4ZEh3aHZnRUNEL0VySnBzTmFraDdNbS9nMXhWanhIcw==',

yt_dlp/extractor/youtube/_base.py

@@ -99,7 +99,7 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'WEB',
-                'clientVersion': '2.20260114.08.00',
+                'clientVersion': '2.20260708.00.00',
             },
         },
         'INNERTUBE_CONTEXT_CLIENT_NAME': 1,
@@ -107,11 +107,12 @@ INNERTUBE_CLIENTS = {
         **WEB_PO_TOKEN_POLICIES,
     },
     # Safari UA returns pre-merged video+audio 144p/240p/360p/720p/1080p HLS formats
+    # Since 2026.07, HLS formats are only returned with some logged-in or "trusted" sessions
     'web_safari': {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'WEB',
-                'clientVersion': '2.20260114.08.00',
+                'clientVersion': '2.20260708.00.00',
                 'userAgent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15,gzip(gfe)',
             },
         },
@@ -123,7 +124,7 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'WEB_EMBEDDED_PLAYER',
-                'clientVersion': '1.20260115.01.00',
+                'clientVersion': '2.20260708.00.00',
             },
         },
         'INNERTUBE_CONTEXT_CLIENT_NAME': 56,
@@ -134,7 +135,7 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'WEB_REMIX',
-                'clientVersion': '1.20260114.03.00',
+                'clientVersion': '1.20260707.12.00',
             },
         },
         'INNERTUBE_CONTEXT_CLIENT_NAME': 67,
@@ -164,7 +165,7 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'WEB_CREATOR',
-                'clientVersion': '1.20260114.05.00',
+                'clientVersion': '1.20260708.06.00',
             },
         },
         'INNERTUBE_CONTEXT_CLIENT_NAME': 62,
@@ -193,9 +194,9 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'ANDROID',
-                'clientVersion': '21.02.35',
+                'clientVersion': '21.26.364',
                 'androidSdkVersion': 30,
-                'userAgent': 'com.google.android.youtube/21.02.35 (Linux; U; Android 11) gzip',
+                'userAgent': 'com.google.android.youtube/21.26.364 (Linux; U; Android 11) gzip',
                 'osName': 'Android',
                 'osVersion': '11',
             },
@@ -223,6 +224,7 @@ INNERTUBE_CLIENTS = {
     },
     # "Made for kids" videos aren't available with this client
     # Using a clientVersion>1.65 may return SABR streams only
+    # Since 2026.07, intermittent/selective POT enforcement has been observed for non-HLS formats
     'android_vr': {
         'INNERTUBE_CONTEXT': {
             'client': {
@@ -238,6 +240,24 @@ INNERTUBE_CLIENTS = {
         },
         'INNERTUBE_CONTEXT_CLIENT_NAME': 28,
         'REQUIRE_JS_PLAYER': False,
+        'GVS_PO_TOKEN_POLICY': {
+            StreamingProtocol.HTTPS: GvsPoTokenPolicy(
+                required=True,
+                recommended=True,
+                not_required_with_player_token=True,
+            ),
+            StreamingProtocol.DASH: GvsPoTokenPolicy(
+                required=True,
+                recommended=True,
+                not_required_with_player_token=True,
+            ),
+            StreamingProtocol.HLS: GvsPoTokenPolicy(
+                required=False,
+                recommended=True,
+                not_required_with_player_token=True,
+            ),
+        },
+        'PLAYER_PO_TOKEN_POLICY': PlayerPoTokenPolicy(required=False, recommended=True),
     },
     # iOS clients have HLS live streams. Setting device model to get 60fps formats.
     # See: https://github.com/TeamNewPipe/NewPipeExtractor/issues/680#issuecomment-1002724558
@@ -245,10 +265,10 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'IOS',
-                'clientVersion': '21.02.3',
+                'clientVersion': '21.26.4',
                 'deviceMake': 'Apple',
                 'deviceModel': 'iPhone16,2',
-                'userAgent': 'com.google.ios.youtube/21.02.3 (iPhone16,2; U; CPU iOS 18_3_2 like Mac OS X;)',
+                'userAgent': 'com.google.ios.youtube/21.26.4 (iPhone16,2; U; CPU iOS 18_3_2 like Mac OS X;)',
                 'osName': 'iPhone',
                 'osVersion': '18.3.2.22D82',
             },
@@ -270,13 +290,29 @@ INNERTUBE_CLIENTS = {
         'PLAYER_PO_TOKEN_POLICY': PlayerPoTokenPolicy(required=False, recommended=True),
         'REQUIRE_JS_PLAYER': False,
     },
+    # "Made for kids" videos aren't available with this client
+    'visionos': {
+        'INNERTUBE_CONTEXT': {
+            'client': {
+                'clientName': 'VISIONOS',
+                'clientVersion': '1.02',
+                'deviceMake': 'Apple',
+                'deviceModel': 'RealityDevice17,1',
+                'userAgent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7_3) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/26.0 Safari/605.1.15',
+                'osName': 'visionOS',
+                'osVersion': '26.5.23O471',
+            },
+        },
+        'INNERTUBE_CONTEXT_CLIENT_NAME': 101,
+        'REQUIRE_JS_PLAYER': False,
+    },
     # mweb has 'ultralow' formats
     # See: https://github.com/yt-dlp/yt-dlp/pull/557
     'mweb': {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'MWEB',
-                'clientVersion': '2.20260115.01.00',
+                'clientVersion': '2.20260708.05.00',
                 # mweb previously did not require PO Token with this UA
                 'userAgent': 'Mozilla/5.0 (iPad; CPU OS 16_7_10 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Mobile/15E148 Safari/604.1,gzip(gfe)',
             },
@@ -307,7 +343,7 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'TVHTML5',
-                'clientVersion': '7.20260114.12.00',
+                'clientVersion': '7.20260707.07.00',
                 # See: https://github.com/youtube/cobalt/blob/main/cobalt/browser/user_agent/user_agent_platform_info.cc#L506
                 'userAgent': 'Mozilla/5.0 (ChromiumStylePlatform) Cobalt/25.lts.30.1034943-gold (unlike Gecko), Unknown_TV_Unknown_0/Unknown (Unknown, Unknown)',
             },
@@ -319,12 +355,11 @@ INNERTUBE_CLIENTS = {
         'INNERTUBE_CONTEXT': {
             'client': {
                 'clientName': 'TVHTML5',
-                'clientVersion': '5.20260114',
+                'clientVersion': '5.20260707',
                 'userAgent': 'Mozilla/5.0 (ChromiumStylePlatform) Cobalt/Version',
             },
         },
         'INNERTUBE_CONTEXT_CLIENT_NAME': 7,
-        'REQUIRE_AUTH': True,
         'SUPPORTS_COOKIES': True,
     },
     'tv_simply': {

yt_dlp/extractor/youtube/_video.py

@@ -28,6 +28,7 @@ from .jsc._director import initialize_jsc_director
 from .jsc.provider import JsChallengeRequest, JsChallengeType, NChallengeInput, SigChallengeInput
 from .pot._director import initialize_pot_director
 from .pot.provider import PoTokenContext, PoTokenRequest
+from ...networking import HEADRequest
 from ...utils import (
     NO_DEFAULT,
     ExtractorError,
@@ -139,13 +140,13 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
     _RETURN_TYPE = 'video'  # XXX: How to handle multifeed?
 
     _SUBTITLE_FORMATS = ('json3', 'srv1', 'srv2', 'srv3', 'ttml', 'srt', 'vtt')
-    _DEFAULT_CLIENTS = ('android_vr', 'web_safari')
+    _DEFAULT_CLIENTS = ('visionos', 'android_vr', 'web')
-    _DEFAULT_JSLESS_CLIENTS = ('android_vr',)
+    _DEFAULT_JSLESS_CLIENTS = ('visionos', 'android_vr')
-    _DEFAULT_AUTHED_CLIENTS = ('tv_downgraded', 'web_safari')
+    _DEFAULT_AUTHED_CLIENTS = ('tv_downgraded', 'web')
     # Premium does not require POT (except for subtitles)
-    _DEFAULT_PREMIUM_CLIENTS = ('tv_downgraded', 'web_creator')
+    _DEFAULT_PREMIUM_CLIENTS = ('tv_downgraded', 'web_creator', 'web')
     _WEBPAGE_CLIENTS = ('web', 'web_safari')
-    _DEFAULT_WEBPAGE_CLIENT = 'web_safari'
+    _DEFAULT_WEBPAGE_CLIENT = 'web'
 
     _GEO_BYPASS = False
 
@@ -2036,8 +2037,11 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                 last_seq = last_seq_cache[cache_key]
             else:
                 try:
-                    urlh = self._request_webpage(base_url, None, note=False, errnote=False, fatal=False)
+                    urlh = self._request_webpage(
-                except ExtractorError:
+                        HEADRequest(base_url), None,
+                        note=False, errnote='Fragment request failed')
+                except ExtractorError as e:
+                    self.write_debug(e.msg)
                     urlh = None
                 last_seq = try_get(urlh, lambda x: int_or_none(x.headers['X-Head-Seqnum']))
                 if urlh:
@@ -2900,6 +2904,10 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
     def _is_unplayable(player_response):
         return traverse_obj(player_response, ('playabilityStatus', 'status')) == 'UNPLAYABLE'
 
+    @staticmethod
+    def _is_error_response(player_response):
+        return traverse_obj(player_response, ('playabilityStatus', 'status')) == 'ERROR'
+
     def _extract_player_response(self, client, video_id, webpage_ytcfg, player_ytcfg, player_url, initial_pr, visitor_data, data_sync_id, po_token):
         headers = self.generate_api_headers(
             ytcfg=player_ytcfg,
@@ -3132,13 +3140,14 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
                     prs.append(pr)
 
             if (
-                # Is this a "made for kids" video that can't be downloaded with android_vr?
+                # Is this a "made for kids" video that can't be downloaded with android_vr/visionos?
-                client == 'android_vr' and self._is_unplayable(pr)
+                client in {'android_vr', 'visionos'}
+                and (self._is_unplayable(pr) or self._is_error_response(pr))
                 and webpage and 'made for kids' in webpage
                 # ...and is a JS runtime is available?
                 and any(p.is_available() for p in self._jsc_director.providers.values())
             ):
-                append_client('web_embedded')
+                append_client('tv_downgraded')
 
             # web_embedded can work around age-gate and age-verification for some embeddable videos
             if self._is_agegated(pr) and variant != 'web_embedded':