Commit Graph
80 Commits
Author SHA1 Message Date
coletdjnzandGitHub 141a8dff98 [extractor/youtube] Fix comment loop detection for pinned comments (#6714)
Pinned comments may repeat a second time - this is expected.

Fixes https://github.com/yt-dlp/yt-dlp/issues/6712

Authored by: coletdjnz
2023-04-06 07:44:22 +00:00
coletdjnzandGitHub 607510b9f2 [extractor/youtube] Handle incomplete initial data from watch page (#6510)
Authored by: coletdjnz
2023-03-13 01:43:37 +00:00
coletdjnzandGitHub 7f51861b18 [extractor/youtube] Detect and break on looping comments (#6301)
Fixes https://github.com/yt-dlp/yt-dlp/issues/6290

Authored by: coletdjnz
2023-03-01 07:56:53 +00:00
MatthewandGitHub 8300774c4a Add --enable-file-urls (#5917)
Closes https://github.com/yt-dlp/yt-dlp/issues/3675

Authored by: coletdjnz
2023-01-02 06:05:13 +00:00
MatthewandGitHub e756f45ba0 Improve handling for overriding extractors with plugins (#5916)
* Extractors replaced with plugin extractors now show in debug output
* Better testcase handling
* Added documentation
Authored by: coletdjnz, pukkandan
2023-01-02 04:55:11 +00:00
coletdjnz 3e01ce744a [extractor/generic] Use Accept-Encoding: identity for initial request
The existing comment seems to imply this was the desired behavior from the beginning.

Partial fix for https://github.com/yt-dlp/yt-dlp/issues/5855, https://github.com/yt-dlp/yt-dlp/issues/5851, https://github.com/yt-dlp/yt-dlp/issues/4748
2023-01-01 18:41:35 +13:00
MatthewandGitHub 8e40b9d1ec Improve plugin architecture (#5553)
to make plugins easier to develop and use:
* Plugins are now loaded as namespace packages.
* Plugins can be loaded in any distribution of yt-dlp (binary, pip, source, etc.).
* Plugin packages can be installed and managed via pip, or dropped into any of the documented locations.
* Users do not need to edit any code files to install plugins.
* Backwards-compatible with previous plugin architecture.

As a side-effect, yt-dlp will now search in a few more locations for config files.

Closes https://github.com/yt-dlp/yt-dlp/issues/1389

Authored by: flashdagger, coletdjnz, pukkandan, Grub4K
Co-authored-by: Marcel <flashdagger@googlemail.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
2023-01-01 04:29:22 +00:00
MatthewandGitHub c733555106 [extractor/youtube:tab] Extract metadata from channel items (#5569)
Authored by: coletdjnz
2022-12-12 23:08:14 +00:00
MatthewandGitHub 3f5c216969 [extractor/nzherald] Support new video embed (#5493)
Authored by: coletdjnz
2022-11-10 21:12:10 +00:00
MatthewandGitHub e72e48c53f [extractor/youtube] Ignore incomplete data error for comment replies (#5490)
When --ignore-errors is used.
Closes https://github.com/yt-dlp/yt-dlp/issues/4669
Authored by: coletdjnz
2022-11-10 06:35:22 +00:00
MatthewandGitHub 0cf643b234 [extractor/youtube] Differentiate between no and disabled comments (#5491)
`comments` and `comment_count` will be set to None, as opposed to 
an empty list and 0, respectively.

Fixes https://github.com/yt-dlp/yt-dlp/issues/5068

Authored by: coletdjnz, pukkandan
2022-11-10 03:33:03 +00:00
MatthewandGitHub 4dc23a8051 [extractor/youtube:tab] Fix video metadata from tabs (#5489)
Closes #5488
Authored by: coletdjnz
2022-11-10 08:14:12 +05:30
MatthewandGitHub d9df9b4919 [extractor/unsupported] Raise error on known DRM-only sites (#5483)
Authored by: coletdjnz
2022-11-09 09:09:13 +00:00
MatthewandGitHub 86973308cd [extractor/youtube:tab] Update tab handling for redesign (#5439)
Closes #5432, #5430, #5419
Authored by: coletdjnz, pukkandan
2022-11-09 14:28:44 +05:30
MatthewandGitHub ac8e69dd32 Do not backport Python 3.10 SSL configuration for LibreSSL (#5464)
Until further investigation.

Fixes regression in https://github.com/yt-dlp/yt-dlp/commit/5b9f253fa0aee996cf1ed30185d4b502e00609c4

Authored by: coletdjnz
2022-11-06 20:30:55 +00:00
MatthewandGitHub 5b9f253fa0 Backport SSL configuration from Python 3.10 (#5437)
Partial fix for https://github.com/yt-dlp/yt-dlp/pull/5294#issuecomment-1289363572, https://github.com/yt-dlp/yt-dlp/issues/4627

Authored by: coletdjnz
2022-11-06 22:07:23 +05:30
MatthewandGitHub 6141346d18 [extractor/youtube] Update playlist metadata extraction for new layout (#5376)
Fixes https://github.com/yt-dlp/yt-dlp/issues/5373

Authored by: coletdjnz
2022-11-06 05:25:31 +00:00
coletdjnz e63faa101c [extractor/youtube] Fix live_status extraction for playlist videos
Regression in https://github.com/yt-dlp/yt-dlp/commit/867c66ff97b0639485a2b6ebc28f2e0df0bf8187

Authored by: coletdjnz
2022-10-27 17:36:54 +13:00
MatthewandGitHub 217753f4aa [extractor/YoutubeWebArchive] Improve metadata extraction (#4968)
Closes https://github.com/yt-dlp/yt-dlp/issues/4574

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2022-10-17 05:46:24 +00:00
MatthewandGitHub 6dca2aa66d [extractor/generic:quoted-html] Add extractor (#5213)
Extracts embeds from escaped HTML within `data-html` attribute.
Related: https://github.com/ytdl-org/youtube-dl/issues/21294, https://github.com/yt-dlp/yt-dlp/pull/5121

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2022-10-14 04:32:52 +00:00
MatthewandGitHub 4c9a1a3ba5 [extractor/wordpress:mb.miniAudioPlayer] Add embed extractor (#5087)
Closes https://github.com/yt-dlp/yt-dlp/issues/4994

Authored by: coletdjnz
2022-10-09 05:55:26 +00:00
MatthewandGitHub 867c66ff97 [extractor/youtube] Extract concurrent view count for livestreams (#5152)
Adds new field `concurrent_view_count`
Closes https://github.com/yt-dlp/yt-dlp/issues/4843

Authored by: coletdjnz
2022-10-07 07:00:40 +00:00
coletdjnzandGitHub 34859e4b32 [extractor/onenewsnz] Add extractor (#5088)
Authored by: coletdjnz
2022-10-04 09:44:57 +05:30
coletdjnzandGitHub 1e0daeb314 [extractor/24tv.ua] Add extractors (#5121)
Closes #4287
Authored by: coletdjnz
2022-10-04 08:59:29 +05:30
coletdjnzandGitHub 7474e4531e [extractor/AmazonStore] Fix JSON extraction (#5111)
Fixes https://github.com/yt-dlp/yt-dlp/issues/5110

Authored by: coletdjnz
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2022-10-03 19:40:49 +00:00
coletdjnzandGitHub d3a3d7f0cc [extractor/JWPlatform] Fix extractor (#5112)
Fix bitrate and filesize extraction and support embeds with unquoted urls.

Related: #5106 

Authored by: coletdjnz
2022-10-03 19:37:48 +00:00
coletdjnz f1aae71568 [extractor/rcs] Fix embed extraction
Fixes https://github.com/yt-dlp/yt-dlp/issues/5076

Authored by: coletdjnz
2022-09-30 14:02:20 +13:00
coletdjnzandGitHub c9eba8075f [extractor/wordpress:playlist] Add generic embed extractor (#5012)
Fixes https://github.com/yt-dlp/yt-dlp/issues/4955

Authored by: coletdjnz
2022-09-28 06:37:12 +00:00
coletdjnzandGitHub 0ca0f88121 [extractor/heise] Fix extractor (#5029)
Fixes https://github.com/yt-dlp/yt-dlp/issues/1520
Authored by: coletdjnz
2022-09-26 00:58:06 +00:00
coletdjnz d42763a443 [extractor/rutube] Fix _EMBED_REGEX
Closes https://github.com/yt-dlp/yt-dlp/issues/4797

Authored by: coletdjnz
2022-09-24 17:44:26 +12:00
coletdjnzandGitHub 3c757d5ed2 [extractor/wistia] Add support for channels (#4819)
Fixes https://github.com/yt-dlp/yt-dlp/issues/4748
Related: https://github.com/yt-dlp/yt-dlp/issues/4985

Authored by: coletdjnz
2022-09-23 21:52:11 +00:00
coletdjnz 32972518da [extractor/telegraaf] Use mobile GraphQL API endpoint
Workaround for Cloudflare 403
Fixes https://github.com/yt-dlp/yt-dlp/issues/5000
Authored by: coletdjnz
2022-09-23 12:10:35 +12:00
coletdjnzandGitHub 80eb0bd9b9 [extractor/youtube] Add support for Shorts audio pivot feed (#4932)
This feed shows Shorts using the audio of a given video. 

ytshortsap: prefix can be used as a shortcut until YouTube
implements an official view. 

Closes #4911
Authored by: coletdjnz
2022-09-22 05:39:02 +00:00
coletdjnz 0831d95c46 [extractor/BiliIntl] Support uppercase lang in _VALID_URL
Seen in some rare cases
Authored by: coletdjnz
2022-09-10 10:06:48 +12:00
coletdjnzandGitHub c26f9b991a [extractor/youtube] Support changing extraction language (#4470)
Adds `--extractor-args youtube:lang=<supported lang code>` extractor arg to prefer translated fields (e.g. title and description) of that language, if available, for all YouTube extractors. See README or error message for list of supported language codes.

Closes https://github.com/yt-dlp/yt-dlp/issues/387

Authored by: coletdjnz
2022-09-09 05:16:46 +00:00
coletdjnz 3ffb2f5bea [extractor/youtube] Fix video like count extraction
Support new combined button layout
Authored by: coletdjnz
2022-09-09 12:34:39 +12:00
coletdjnzandGitHub aa824dd10b [extractor/mediaworksnzvod] Add extractor (#4817)
Authored by: coletdjnz
2022-09-03 03:19:48 +00:00
coletdjnzandGitHub 1ff88b7aec [extractor/youtube] Add no-youtube-prefer-utc-upload-date compat option (#4771)
This option reverts https://github.com/yt-dlp/yt-dlp/commit/992f9a730b49fd36fc422be8d802f98ebcdce418 and https://github.com/yt-dlp/yt-dlp/commit/17322130a954577bb03b833d5c435638e51e19f2 to prefer the non-UTC upload date in microformats.

Authored by: coletdjnz, pukkandan
2022-09-01 10:02:28 +00:00
coletdjnzandGitHub 50ac0e5416 [extractor/youtube] Use device-specific user agent (#4770)
Thwart latest fingerprinting attempt (see https://github.com/iv-org/invidious/issues/3230#issuecomment-1226887639)

Authored by: coletdjnz
2022-08-28 22:59:54 +00:00
coletdjnzandGitHub cea4b857f0 [patreon] Ignore erroneous media attachments (#4638)
Fixes https://github.com/yt-dlp/yt-dlp/issues/4608
Authored by: coletdjnz
2022-08-13 00:25:20 +00:00
coletdjnz c7dcf0b31e [extractor/youtube] Add androidSdkVersion parameter to Android Innertube clients
Required to prevent YouTube returning a bad player response in some cases.

See: https://github.com/yt-dlp/yt-dlp/pull/4593, https://github.com/TeamNewPipe/NewPipe/issues/8713, https://github.com/iv-org/invidious/issues/3230, https://github.com/Tyrrrz/YoutubeExplode/issues/647

Authored by: coletdjnz
2022-08-08 12:03:10 +12:00
coletdjnzandGitHub a3e9642116 [extractor/youtube] Prevent redirect to unwanted videos (#4593)
Example: https://www.youtube.com/watch?v=aQvGIIdgFDM

Authored by: coletdjnz
2022-08-07 19:13:20 +05:30
coletdjnz a0c830f488 [extractor/youtube] Bump Innertube client versions
YouTube may be requiring new versions soon. See https://github.com/iv-org/invidious/issues/3230, https://github.com/TeamNewPipe/NewPipe/issues/8713

Authored by: coletdjnz
2022-08-02 19:02:05 +12:00
coletdjnzandGitHub 871a8929bc [extractor/archiveorg] Improve handling of formats (#4461)
* Ignore private formats if not logged in (fixes https://github.com/yt-dlp/yt-dlp/issues/3832)
* Prefer original formats
* Support mpg formats

Authored by: coletdjnz, pukkandan
2022-07-29 05:09:36 +00:00
coletdevandGitHub 4f08e58655 [extractor/patreon] Fix and improve extractors (#4398)
* Add workaround for 403s - Fixes https://github.com/yt-dlp/yt-dlp/issues/3631
* Support m3u8 post file videos - Fixes https://github.com/yt-dlp/yt-dlp/issues/2277
* Raise useful error messages - Fixes https://github.com/yt-dlp/yt-dlp/issues/2914
* `--write-comments` support

Authored by: coletdjnz, pukkandan
2022-07-22 22:23:54 +00:00
coletdevandGitHub 379a4f161d [utils] Fix inconsistent default handling between HTTP and HTTPS requests (#4158)
Default headers such as `Content-Type` were only being added for HTTPS requests among other handling.

Fixes bug in https://github.com/ytdl-org/youtube-dl/commit/be4a824d74add1a3b78b8244dff12f4f078f168a

Authored-by: coletdjnz
2022-06-24 03:29:28 +00:00
coletdevandGitHub f0bc6e2019 [extractor] Add default parameter to _search_json (#4057)
Authored by: pukkandan, coletdjnz
2022-06-18 17:55:18 -07:00
coletdjnz 4ce05f5759 [extractor/youtube] Fix live chat for videos with content warning
Fixes #4051
Authored by: coletdjnz
2022-06-12 17:56:50 +12:00
coletdevandGitHub 2ae778b8fc [extractor/youtube] Add innertube_host and innertube_key extractor args (#3916)
Allows user to override Innertube API host or key for all requests
Authored by: coletdjnz
2022-06-08 22:18:01 +00:00
coletdjnz 65141660ab [extractor/youtube] Fix bug in b7c47b7438
Closes #3997

Authored by: coletdjnz
2022-06-07 12:26:36 +12:00
coletdevandGitHub c7a7baaa13 [extractor/youtube] Fix :ytnotifications extractor (#3775)
Still some issues, see https://github.com/yt-dlp/yt-dlp/pull/3775

Authored by: coletdjnz
2022-06-03 07:04:39 +00:00
coletdjnz ee27297f82 [extractor/youtube] Fix initial player response extraction
Authored by: pukkandan, coletdjnz
2022-05-29 19:54:22 +12:00
coletdevandGitHub 2c6dcb65fb [utils] Send HTTP/1.1 ALPN extension (#3889)
Some servers may reject requests if not sent (e.g. fingerprinting)

Fixes #3878

Authored by: coletdjnz
2022-05-28 03:46:36 +00:00
coletdjnz 5faf6528fb [http] Fix bug in retrying on read timeout in py < 3.10
socket.timeout is not an alias of TimeoutError in py < 3.10
Fixes bug in https://github.com/yt-dlp/yt-dlp/commit/a2e77303e3385da640a0904cd6cb76235fa9691b
Authored-by: coletdjnz
2022-05-17 09:17:37 +12:00
bb58c9ed5c Add support for SSL client certificate authentication (#3435)
Adds `--client-certificate`, `--client-certificate-key`, `--client-certificate-password`

Authored-by: coletdjnz
Co-authored-by: df <fieldhouse@gmx.net>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
2022-05-02 07:59:45 +00:00
coletdevandGitHub afac4caa7d Fix redirect HTTP method handling (#3577)
Authored by: coletdjnz
2022-05-01 20:40:26 -07:00
coletdevandGitHub 6e634cbe42 [youtube] Add YoutubeStoriesIE (#3362)
Get channel stories with `ytstories:<channel UCID>`

Authored-by: coletdjnz
2022-05-01 06:46:28 +00:00
coletdjnz affc4fefea [youtube] Fix episode metadata extraction 2022-04-15 16:22:03 +12:00
coletdevandGitHub 61d3665d9d [youtube] Fix uploader for collaborative playlists (#3332)
Authored by: coletdjnz
2022-04-07 01:11:16 -07:00
coletdevandGitHub ad210f4fd4 [youtube:search] Support hashtag entries (#3265)
Authored-by: coletdjnz
2022-04-02 06:11:14 +00:00
coletdjnz c8e856a551 [web.archive:youtube] Make CDX API requests non-fatal
Partial fix for https://github.com/yt-dlp/yt-dlp/issues/3278
Authored-by: coletdjnz
2022-04-02 19:07:13 +13:00
coletdevandGitHub 1c1b2f96ae [youtube:tab] Fix duration extraction for shorts (#3171)
Related: https://github.com/TeamNewPipe/NewPipe/issues/8034
Authored-by: coletdjnz
2022-03-28 00:49:42 +00:00
coletdevandGitHub fd2ad7cb24 [youtube:tab] Return shorts url if video is a short (#3168)
Allows filtering out shorts from feeds with `--match-filter`
Closes #3165
Authored-by: coletdjnz
2022-03-27 05:20:25 +00:00
coletdevandGitHub af14914baa Remove Accept-Encoding header from std_headers (#3153)
This should be set by each downloader to what it supports.
Fixes https://github.com/yt-dlp/yt-dlp/issues/3142
Authored-by: coletdjnz
2022-03-23 07:47:02 +00:00
coletdevandGitHub d5820461e8 Use certificates from certifi if installed (#3115)
Fixes #3102 and most `CERTIFICATE_VERIFY_FAILED` issues

Authored by: coletdjnz
2022-03-22 16:26:55 -07:00
coletdevandGitHub 8a23db9519 [wget] Fix proxy (#3152)
Upstream PR: https://github.com/ytdl-org/youtube-dl/pull/29343
Authored-by: kikuyan, coletdjnz
2022-03-22 14:24:27 -07:00
coletdevandGitHub e6552207da [panopto] Improve subtitle extraction and support slides (#3009)
Related: #1946, #2908
Authored-by: coletdjnz
2022-03-18 22:19:36 +00:00
coletdevandGitHub a2e77303e3 [downloader/http] Retry on more errors (#3065)
Closes #3056, #2071
Related: #3034, #2969
Authored-by: coletdjnz
2022-03-18 22:10:20 +00:00
coletdjnz 5dbc77df26 [youtube:api] Prefer minified JSON response
Authored-by: coletdjnz
2022-03-16 09:29:15 +13:00
coletdevandGitHub b3edc8068e [downloader/mhtml] Fix fragments with absolute urls (#3044)
Authored-by: coletdjnz
2022-03-13 22:03:40 +00:00
coletdevandGitHub 17322130a9 [youtube] Improve video upload date handling (#3029)
* Don't prefer UTC upload date for past live streams/premieres
* Improve regex (fixes a regression)

Authored-by: coletdjnz
2022-03-13 22:02:44 +00:00
coletdjnz a825ffbffa [extractor] Support merging subtitles with data
Authored-by: coletdjnz
2022-03-12 11:22:28 +13:00
coletdevandGitHub e248be3319 [panopto] Add extractors (#2908)
Based on https://github.com/ytdl-org/youtube-dl/pull/13449
Closes #1946
Authored by: coletdjnz, kmark
2022-03-08 13:00:57 -08:00
coletdevandGitHub 4390d5ec12 Add brotli content-encoding support (#2433)
Authored by: coletdjnz
2022-03-08 08:44:05 -08:00
coletdevandpukkandan 992f9a730b [youtube] Prefer UTC upload date for videos (#2223)
Except for live/scheduled streams/premieres. 
Closes #1881
Related: #2402 
Authored-by: coletdjnz
2022-03-08 12:58:19 +05:30
coletdevandGitHub ded9f32667 [extractor] Support --mark-watched without _NETRC_MACHINE (#2939)
Authored by: coletdjnz
2022-03-03 23:27:09 -08:00
coletdevandGitHub 7c219ea601 [youtube:tab] Follow redirect to regional channel (#2884)
Closes #2694
Authored by: coletdjnz
2022-02-28 21:08:19 -08:00
coletdevandGitHub 60f393e48b [youtube] Ensure subtitle urls are absolute (#2765)
Closes #2755

Authored by: coletdjnz
2022-02-13 13:36:01 -08:00
coletdjnz af5c1c553e [youtube] Fix search extractor
Regression introduced in https://github.com/yt-dlp/yt-dlp/commit/16aa9ea41dd0a575c6c2d6945d3f8fe3bb42b327. Closes #2628
Authored-by: coletdjnz
2022-02-04 10:32:56 +13:00
coletdjnz 87e8e8a7d0 [youtube:api] Do not use seek when reading HTTPError response
Authored-by: coletdjnz
2022-01-23 19:11:32 +13:00