Compare commits

...

24 Commits

Author SHA1 Message Date
LuanRT
07f02d0dc1 chore: remove unnecessary variable 2022-01-30 20:47:26 -03:00
LuanRT
b2afa86744 chore: update deps 2022-01-30 20:43:29 -03:00
LuanRT
a1caa60750 1.2.9 2022-01-30 20:34:18 -03:00
LuanRT
e1dd718832 chore: format code and other minor changes 2022-01-30 20:28:00 -03:00
LuanRT
222bf1e61f fix: ntoken 'translate2' function not being parsed 2022-01-30 20:26:02 -03:00
LuanRT
3b48de20dd fix: oauth identity creds regex no longer working 2022-01-30 20:20:52 -03:00
LuanRT
348d901935 chore: update tests 2022-01-30 20:19:11 -03:00
LuanRT
94b12002ff feat: implement continuation requests for YTMusic 2022-01-18 15:55:07 -03:00
LuanRT
2720e8f251 chore(livechat): remove console.log 2022-01-18 15:52:35 -03:00
LuanRT
a8a1ec2182 fix (tests): video used for tests is no longer available 2022-01-18 15:50:43 -03:00
LuanRT
ee0d1bef40 deps: remove time-to-seconds dependency 2022-01-18 15:46:54 -03:00
LuanRT
5cad39ee44 fix: polling interval missing 2022-01-07 18:59:17 -03:00
LuanRT
e8ca248919 feat: add home feed support 2022-01-07 18:50:00 -03:00
LuanRT
44d09026b5 chore: simplify video details parser 2022-01-07 18:46:29 -03:00
LuanRT
ff044f4216 fix: error polling livechat due to dislikes 2022-01-07 18:45:30 -03:00
LuanRT
8153e6178c fix: subsfeed sections placeholders missing 2022-01-05 16:36:48 -03:00
LuanRT
ee3f1b4638 chore: update examples 2022-01-05 16:31:52 -03:00
LuanRT
86c8a7e0d2 fix: filter out undefined search results 2021-12-31 04:05:38 -03:00
LuanRT
b375ae2f06 chore: fix typo 2021-12-31 03:35:05 -03:00
LuanRT
2ff4b2ea95 test: remove node 12 build 2021-12-31 03:27:54 -03:00
LuanRT
599ab69107 refactor: rewrite inefficient code and add docs 2021-12-31 03:19:58 -03:00
LuanRT
c6c6dc24bd feat: add support for music search 2021-12-31 03:15:59 -03:00
LuanRT
fa2e0724c6 docs: fix a typo 2021-12-22 15:27:26 -03:00
LuanRT
6af689ada6 docs: improve documentation & add unseen notifications example 2021-12-22 15:22:10 -03:00
17 changed files with 849 additions and 456 deletions

View File

@@ -16,7 +16,7 @@ jobs:
strategy:
matrix:
node-version: [ 12.x, 14.x, 15.x]
node-version: [ 14.x, 15.x, 16.x ]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:

View File

@@ -1,24 +1,30 @@
# YouTube.js
<h1 align="center">YouTube.js</h1>
<p align="center"><i>An object-oriented wrapper around the Innertube API, which is what YouTube itself uses.</i><p>
[![Build](https://github.com/LuanRT/YouTube.js/actions/workflows/node.js.yml/badge.svg)](https://github.com/LuanRT/YouTube.js/actions/workflows/node.js.yml)
[![NPM](https://img.shields.io/npm/v/youtubei.js?color=%2335C757)](https://www.npmjs.com/package/youtubei.js)
[![CodeFactor](https://www.codefactor.io/repository/github/luanrt/youtube.js/badge)](https://www.codefactor.io/repository/github/luanrt/youtube.js)
<p align="center">
<img src="https://github.com/LuanRT/YouTube.js/actions/workflows/node.js.yml/badge.svg">
<img src="https://img.shields.io/npm/v/youtubei.js?color=%2335C757">
<img src="https://www.codefactor.io/repository/github/luanrt/youtube.js/badge">
</p>
Innertube is an API used across all YouTube clients, it was [made to simplify](https://gizmodo.com/how-project-innertube-helped-pull-youtube-out-of-the-gu-1704946491) the internal structure of the platform and make it easy to push updates. This library takes advantage of that API, therefore providing a simple & efficient way to interact with YouTube programmatically.
And big thanks to [@gatecrasher777](https://github.com/gatecrasher777/ytcog) for his research on the workings of the Innertube API!
An object-oriented wrapper around the Innertube API, which is what YouTube itself uses. This makes YouTube.js fast, simple & efficient. And big thanks to [@gatecrasher777](https://github.com/gatecrasher777/ytcog) for his research on the workings of the Innertube API!
#### What can it do?
As of now, this is one of the most advanced & stable YouTube libraries out there, here's a short summary of what it can do:
As of now, this is one of the most advanced & stable YouTube libraries out there, here's a short summary of its features:
- Search videos
- Get detailed info about any video
- Fetch live chat & live stats in real time
- Fetch notifications
- Fetch subscriptions feed
- Change notifications preferences for a channel
- Subscribe/Unsubscribe/Like/Dislike/Comment, etc
- Easily sign into your account in an easy & reliable way.
- Last but not least, you can also download videos!
- Get notifications
- Get subscriptions feed
- Change notification preferences for a channel
- Subscribe/Unsubscribe/Like/Dislike/Comment etc
- Easily sign in to any Google Account
- Download videos
Do note that you must be signed-in to perform actions that involve an account, such as commenting, subscribing, sending messages to a live chat, etc.
@@ -38,7 +44,7 @@ npm install youtubei.js
[2. Interactions](https://github.com/LuanRT/YouTube.js#interactions)
[3. Fetching live chats](https://github.com/LuanRT/YouTube.js#fetching-live-chats)
[3. Live chats](https://github.com/LuanRT/YouTube.js#fetching-live-chats)
[4. Downloading videos](https://github.com/LuanRT/YouTube.js#downloading-videos)
@@ -46,21 +52,13 @@ npm install youtubei.js
[6. Disclaimer](https://github.com/LuanRT/YouTube.js#disclaimer)
First of all we're gonna start by initializing the Innertube class:
First of all we're gonna start by initializing the Innertube instance.
And to make things faster, you should do this only once and reuse the Innertube object when needed.
```js
const Innertube = require('youtubei.js');
async function start() {
const youtube = await new Innertube();
//...
}
start();
const youtube = await new Innertube();
```
After this you should be good to go, so let's dive into it!
Doing a simple search:
```js
@@ -489,13 +487,19 @@ const notifications = await youtube.getNotifications();
</p>
</details>
Get unseen notifications count:
```js
const notifications = await youtube.getUnseenNotificationsCount();
```
### Interactions:
---
* Subscribe/Unsubscribe:
```js
const video = await youtube.getDetails(VIDEO_ID_HERE); // this is equivalent to opening the watch page on YouTube
const video = await youtube.getDetails(VIDEO_ID_HERE);
await video.subscribe();
await video.unsubscribe();
@@ -503,7 +507,7 @@ await video.unsubscribe();
* Like/Dislike:
```js
const video = await youtube.getDetails(VIDEO_ID_HERE); // this is equivalent to opening the watch page on YouTube
const video = await youtube.getDetails(VIDEO_ID_HERE);
await video.like();
await video.dislike();
@@ -520,7 +524,7 @@ await video.comment('Haha, nice!');
```js
const video = await youtube.getDetails(VIDEO_ID_HERE);
// Can be: ALL | NONE | PERSONALIZED
// Options: ALL | NONE | PERSONALIZED
await video.setNotificationPref('ALL');
```
@@ -538,7 +542,6 @@ async function start() {
const search = await youtube.search('Some random livestream');
const video = await youtube.getDetails(search.videos[0].id);
// This should only be called if you're sure it's a livestream and that it's still ongoing
const livechat = video.getLivechat();
// Updated stats about the livestream
@@ -563,7 +566,7 @@ Stop fetching the live chat:
livechat.stop();
```
Deleting a message:
Delete a message:
```js
const msg = await livechat.sendMessage('Nice livestream!');
await msg.deleteMessage();
@@ -580,7 +583,9 @@ const Innertube = require('youtubei.js');
async function start() {
const youtube = await new Innertube();
const search = await youtube.search('Looking for life on Mars - documentary');
const stream = youtube.download(search.videos[0].id, {
format: 'mp4', // Optional, ignored when type is set to audio and defaults to mp4, and I recommend to leave it as it is
quality: '360p', // if a video doesn't have a specific quality it'll fall back to 360p, also ignored when type is set to audio
@@ -616,7 +621,7 @@ async function start() {
start();
```
You can also download only a portion of a video by specifying a range:
You can also specify a range:
```js
const stream = youtube.download(VIDEO_ID, {
//...
@@ -634,10 +639,10 @@ stream.cancel();
### Signing-in:
---
This library allows you to sign-in in two different ways:
When signing in to your account, you have two options:
- Using OAuth 2.0, easy, simple & reliable.
- Cookies, usually more complicated to get and unreliable.
- Use OAuth 2.0; easy, simple & reliable.
- Cookies; usually more complicated to get and unreliable.
OAuth:
@@ -685,9 +690,6 @@ async function start() {
start();
```
## Note
Never sign-in with your personal account, you might get banned if you spam (don't ever do that) or simply because YouTube detected unusual activity coming from your account. Also, I'm not responsible if any of that happens to you.
## Contributing
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

View File

@@ -7,18 +7,18 @@ const creds = fs.existsSync(creds_path) && JSON.parse(fs.readFileSync(creds_path
async function start() {
const youtube = await new Innertube();
youtube.on('auth', (data) => {
youtube.ev.on('auth', (data) => {
if (data.status === 'AUTHORIZATION_PENDING') {
console.info(`Hello!\nOn your phone or computer, go to ${data.verification_url} and enter the code ${data.code}`);
} else if (data.status === 'SUCCESS') {
fs.writeFileSync(creds_path, JSON.stringify({ access_token: data.access_token, refresh_token: data.refresh_token, expires: data.expires }));
fs.writeFileSync(creds_path, JSON.stringify(data.credentials));
console.info('Successfully signed-in, enjoy!');
}
});
youtube.on('update-credentials', (data) => {
fs.writeFileSync(creds_path, JSON.stringify({ access_token: data.access_token, refresh_token: data.refresh_token, expires: data.expires }));
youtube.ev.on('update-credentials', (data) => {
fs.writeFileSync(creds_path, JSON.stringify(data.credentials));
console.info('Credentials updated!', data);
});

View File

@@ -5,9 +5,17 @@ const Axios = require('axios');
const Utils = require('./Utils');
const Constants = require('./Constants');
/**
* Performs direct interactions on YouTube.
*
* @param {object} session A valid Innertube session.
* @param {string} engagement_type Type of engagement.
* @param {object} args Engagement arguments.
* @returns {object} { success: boolean, status_code: number } | { success: boolean, status_code: number, message: string }
*/
async function engage(session, engagement_type, args = {}) {
if (!session.logged_in) throw new Error('You are not signed-in');
let data;
switch (engagement_type) {
case 'like/like':
@@ -31,28 +39,41 @@ async function engage(session, engagement_type, args = {}) {
data = {
context: session.context,
commentText: args.text,
createCommentParams: Utils.generateCommentParams(args.video_id)
createCommentParams: Utils.encodeCommentParams(args.video_id)
};
break;
default:
}
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/${engagement_type}${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session, id: args.video_id, data })).catch((error) => error);
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/${engagement_type}${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session, id: args.video_id, data })).catch((error) => error);
if (response instanceof Error) return { success: false, status_code: response.response.status, message: response.message };
return {
success: true,
status_code: response.status
};
}
async function browse(session, action_type) {
/**
* Accesses YouTube's various sections.
*
* @param {object} session A valid Innertube session.
* @param {string} action_type Type of action.
* @returns {object} { success: boolean, status_code: number, data: object } | { success: boolean, status_code: number, message: string }
*/
async function browse(session, action_type, args) {
if (!session.logged_in) throw new Error('You are not signed-in');
let data;
switch (action_type) { // TODO: Handle more actions
case 'home_feed':
data = {
context: session.context,
browseId: 'FEwhat_to_watch'
};
break;
case 'subscriptions_feed':
data = {
context: session.context,
@@ -62,11 +83,12 @@ async function browse(session, action_type) {
default:
}
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/browse${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session })).catch((error) => error);
const client_domain = args.ytmusic && Constants.URLS.YT_MUSIC_URL || Constants.URLS.YT_BASE_URL;
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/browse${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session })).catch((error) => error);
if (response instanceof Error) return { success: false, status_code: response.response.status, message: response.message };
return {
success: true,
status_code: response.status,
@@ -74,17 +96,46 @@ async function browse(session, action_type) {
};
}
async function search(session, args = {}) {
/**
* Performs searches on YouTube.
*
* @param {object} session A valid Innertube session.
* @param {string} client YouTube client: YOUTUBE | YTMUSIC
* @param {object} args Search arguments.
* @returns {object} { success: boolean, status_code: number, data: object } | { success: boolean, status_code: number, message: string }
*/
async function search(session, client, args = {}) {
if (!args.query) throw new Error('No query was provided');
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/search${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify({
context: session.context,
params: Utils.encodeFilter(args.options.period, args.options.duration, args.options.order),
query: args.query
}), Constants.INNERTUBE_REQOPTS({ session })).catch((error) => error);
let data;
switch (client) {
case 'YOUTUBE':
data = {
context: session.context,
params: Utils.encodeFilter(args.options.period, args.options.duration, args.options.order),
query: args.query
};
break;
case 'YTMUSIC':
const yt_music_context = JSON.parse(JSON.stringify(session.context)); // deep copy the context obj so we don't accidentally change it
yt_music_context.client.originalUrl = Constants.URLS.YT_MUSIC_URL;
yt_music_context.client.clientVersion = '1.20211213.00.00';
yt_music_context.client.clientName = 'WEB_REMIX';
data = {
context: yt_music_context,
query: args.query
};
break;
default:
break;
}
const response = await Axios.post(`${client === 'YOUTUBE' && Constants.URLS.YT_BASE_URL || Constants.URLS.YT_MUSIC_URL}/youtubei/v1/search${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session, ytmusic: client === 'YTMUSIC' })).catch((error) => error);
if (response instanceof Error) return { success: false, status_code: response.response.status, message: response.message };
return {
success: true,
status_code: response.status,
@@ -92,9 +143,18 @@ async function search(session, args = {}) {
};
}
/**
* Interacts with YouTube's notification system.
*
* @param {object} session A valid Innertube session.
* @param {string} action_type Type of action.
* @param {object} args Action arguments.
* @returns {object} { success: boolean, status_code: number, data: object } | { success: boolean, status_code: number, message: string }
*/
async function notifications(session, action_type, args = {}) {
if (!session.logged_in) throw new Error('You are not signed-in');
let data;
switch (action_type) {
@@ -119,11 +179,11 @@ async function notifications(session, action_type, args = {}) {
default:
}
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/notification/${action_type}${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session })).catch((error) => error);
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/notification/${action_type}${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session })).catch((error) => error);
if (response instanceof Error) return { success: false, status_code: response.response.status, message: response.message };
if (action_type === 'modify_channel_preference') return { success: true, status_code: response.status };
return {
success: true,
status_code: response.status,
@@ -131,13 +191,28 @@ async function notifications(session, action_type, args = {}) {
};
}
/**
* Interacts with YouTube's livechat system.
*
* @param {object} session A valid Innertube session.
* @param {string} action_type Type of action.
* @param {object} args Action arguments.
* @returns {object} { success: boolean, status_code: number, data: object } | { success: boolean, status_code: number, message: string }
*/
async function livechat(session, action_type, args = {}) {
let data;
switch (action_type) {
case 'live_chat/get_live_chat':
data = {
context: session.context,
continuation: args.ctoken
};
break;
case 'live_chat/send_message':
data = {
context: session.context,
params: Utils.generateMessageParams(args.channel_id, args.video_id),
params: Utils.encodeMessageParams(args.channel_id, args.video_id),
clientMessageId: `ytjs-${Uuid.v4()}`,
richMessage: {
textSegments: [{ text: args.text }]
@@ -155,51 +230,83 @@ async function livechat(session, action_type, args = {}) {
params: args.cmd_params
};
break;
case 'updated_metadata':
data = {
context: session.context,
videoId: args.video_id
};
args.continuation && (data.continuation = args.continuation);
break;
default:
}
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/${action_type}${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session, params: args.params })).catch((error) => error);
if (response instanceof Error) return { success: false, status_code: response.response.status, message: response.message };
return {
success: true,
status_code: response.status,
data: response.data
};
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/${action_type}${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session, params: args.params })).catch((error) => error);
if (response instanceof Error) return { success: false, message: response.message };
return { success: true, data: response.data };
}
/**
* Gets detailed data for a video.
*
* @param {object} session A valid Innertube session.
* @param {object} args Request arguments.
* @returns {object} Video data.
*/
async function getVideoInfo(session, args = {}) {
let response;
!args.is_desktop && (response = await Axios.get(`${Constants.URLS.YT_WATCH_PAGE}?v=${args.id}&t=8s&pbj=1`, Constants.INNERTUBE_REQOPTS({ session, id: args.id, desktop: false })).catch((error) => error)) ||
(response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/player${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(Constants.VIDEO_INFO_REQBODY(args.id, session.sts, session.context)), Constants.INNERTUBE_REQOPTS({ session, id: args.id, desktop: true })).catch((error) => error));
(response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/player${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(Constants.VIDEO_INFO_REQBODY(args.id, session.sts, session.context)), Constants.INNERTUBE_REQOPTS({ session, id: args.id, desktop: true })).catch((error) => error));
if (response instanceof Error) throw new Error(`Could not get video info: ${response.message}`);
return response.data;
}
async function getContinuation(session, info = {}) {
let data = { context: session.context };
info.continuation_token && (data.continuation = info.continuation_token);
if (info.video_id) {
data.videoId = info.video_id;
data.racyCheckOk = true;
data.contentCheckOk = false;
data.autonavState = 'STATE_NONE';
data.playbackContext = {
vis: 0,
lactMilliseconds: '-1'
};
data.captionsRequested = false;
/**
* Requests continuation for previously performed actions.
*
* @param {object} session A valid Innertube session.
* @param {object} args Continuation arguments.
* @returns {object} { success: boolean, status_code: number, data: object } | { success: boolean, status_code: number, message: string }
*/
async function getContinuation(session, args = {}) {
let data = { context: session.context };
args.continuation_token && (data.continuation = args.continuation_token);
if (args.video_id) {
data.videoId = args.video_id;
if (args.ytmusic) {
const yt_music_context = JSON.parse(JSON.stringify(session.context)); // deep copy the context obj so we don't accidentally change it
yt_music_context.client.originalUrl = Constants.URLS.YT_MUSIC_URL;
yt_music_context.client.clientVersion = '1.20211213.00.00';
yt_music_context.client.clientName = 'WEB_REMIX';
data.context = yt_music_context;
data.isAudioOnly = true;
data.tunerSettingValue = 'AUTOMIX_SETTING_NORMAL';
} else {
data.racyCheckOk = true;
data.contentCheckOk = false;
data.autonavState = 'STATE_NONE';
data.playbackContext = {
vis: 0,
lactMilliseconds: '-1'
}
data.captionsRequested = false;
}
}
const response = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/next${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session })).catch((error) => error);
const client_domain = args.ytmusic && Constants.URLS.YT_MUSIC_URL || Constants.URLS.YT_BASE_URL;
const response = await Axios.post(`${client_domain}/youtubei/v1/next${session.logged_in && session.cookie.length < 1 ? '' : `?key=${session.key}`}`,
JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session, ytmusic: args.ytmusic })).catch((error) => error);
if (response instanceof Error) return { success: false, status_code: response.response.status, message: response.message };
return {
success: true,
status_code: response.status,

View File

@@ -5,6 +5,7 @@ const Utils = require('./Utils');
module.exports = {
URLS: {
YT_BASE_URL: 'https://www.youtube.com',
YT_MUSIC_URL: 'https://music.youtube.com',
YT_MOBILE_URL: 'https://m.youtube.com',
YT_WATCH_PAGE: 'https://m.youtube.com/watch'
},
@@ -46,6 +47,9 @@ module.exports = {
},
INNERTUBE_REQOPTS: (info) => {
info.desktop === undefined && (info.desktop = true);
const origin = info.ytmusic && 'https://music.youtube.com' ||
info.desktop && 'https://www.youtube.com' || 'https://m.youtube.com';
let req_opts = {
params: info.params || {},
headers: {
@@ -58,8 +62,8 @@ module.exports = {
'x-youtube-client-name': info.desktop ? 1 : 2,
'x-youtube-client-version': info.session.context.client.clientVersion,
'x-youtube-chrome-connected': 'source=Chrome,mode=0,enable_account_consistency=true,supervised=false,consistency_enabled_by_default=false',
'x-origin': info.desktop ? 'https://www.youtube.com' : 'https://m.youtube.com',
'origin': info.desktop ? 'https://www.youtube.com' : 'https://m.youtube.com',
'x-origin': origin,
'origin': origin,
}
};
@@ -91,11 +95,24 @@ module.exports = {
videoId: id
};
},
METADATA_KEYS: [
'embed', 'view_count', 'average_rating',
'length_seconds', 'channel_id', 'channel_url',
'external_channel_id', 'is_live_content', 'is_family_safe',
'is_unlisted', 'is_private', 'has_ypc_metadata',
'category', 'owner_channel_name', 'publish_date',
'upload_date', 'keywords', 'available_countries',
'owner_profile_url'
],
BLACKLISTED_KEYS: [
'is_owner_viewing', 'is_unplugged_corpus',
'is_crawlable', 'allow_ratings', 'author'
],
BASE64_DIALECT: {
NORMAL: 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_'.split(''),
REVERSE: '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ-_'.split('')
},
FUNCS_REGEX: /d\.push\(e\)|d\.reverse\(\)|d\[0\]\)\[0\]\)|f=d\[0];d\[0\]|d\.length;d\.splice\(e,1\)|function\(\){for\(var|function\(d,e,f\){var h=f|function\(d\){for\(var|reverse\(\)\.forEach|unshift\(d\.pop\(\)\)|function\(d,e\){for\(var f/,
FUNCS_REGEX: /d\.push\(e\)|d\.reverse\(\)|d\[0\]\)\[0\]\)|f=d\[0];d\[0\]|d\.length;d\.splice\(e,1\)|function\(\){for\(var|function\(d,e,f\){var k=f|function\(d\){for\(var|reverse\(\)\.forEach|unshift\(d\.pop\(\)\)|function\(d,e\){for\(var f/,
FUNCS: {
PUSH: 'd.push(e)',
REVERSE_1: 'd.reverse()',
@@ -107,90 +124,16 @@ module.exports = {
ROTATE_2: 'unshift(d.pop())',
BASE64_DIA: 'function(){for(var',
TRANSLATE_1: 'function(d,e){for(var f',
TRANSLATE_2: 'function(d,e,f){var h=f'
TRANSLATE_2: 'function(d,e,f){var k=f'
},
// Helper functions, felt like Utils.js wasn't the right place for them:
// Just a helper function, felt like Utils.js wasn't the right place for it:
formatNTransformData: (data) => {
return data
.replace(/function\(d,e\)/g, '"function(d,e)').replace(/function\(d\)/g, '"function(d)')
.replace(/function\(\)/g, '"function()').replace(/function\(d,e,f\)/g, '"function(d,e,f)')
.replace(/\[function\(d,e,f\)/g, '["function(d,e,f)')
.replace(/,b,/g, ',"b",').replace(/,b/g, ',"b"').replace(/b,/g, '"b",').replace(/b]/g, '"b"]')
.replace(/\[b/g, '["b"').replace(/}]/g, '"]').replace(/},/g, '}",').replace(/""/g, '')
.replace(/length]\)}"/g, 'length])}');
},
formatVideoData: (data, context, is_desktop) => {
let video_details = {};
let metadata = {};
if (is_desktop) {
metadata.embed = data.microformat.playerMicroformatRenderer.embed;
metadata.view_count = parseInt(data.videoDetails.viewCount);
metadata.average_rating = data.videoDetails.averageRating;
metadata.length_seconds = data.microformat.playerMicroformatRenderer.lengthSeconds;
metadata.channel_id = data.videoDetails.channelId;
metadata.channel_url = data.microformat.playerMicroformatRenderer.ownerProfileUrl;
metadata.external_channel_id = data.microformat.playerMicroformatRenderer.externalChannelId;
metadata.is_live_content = data.videoDetails.isLiveContent;
metadata.is_family_safe = data.microformat.playerMicroformatRenderer.isFamilySafe;
metadata.is_unlisted = data.microformat.playerMicroformatRenderer.isUnlisted;
metadata.is_private = data.videoDetails.isPrivate;
metadata.has_ypc_metadata = data.microformat.playerMicroformatRenderer.hasYpcMetadata;
metadata.category = data.microformat.playerMicroformatRenderer.category;
metadata.channel_name = data.microformat.playerMicroformatRenderer.ownerChannelName;
metadata.publish_date = data.microformat.playerMicroformatRenderer.publishDate || 'N/A';
metadata.upload_date = data.microformat.playerMicroformatRenderer.uploadDate || 'N/A';
metadata.keywords = data.videoDetails.keywords || [];
metadata.available_qualities = [...new Set(data.streamingData.adaptiveFormats.filter(v => v.qualityLabel).map(v => v.qualityLabel).sort((a, b) => +a.replace(/\D/gi, '') - +b.replace(/\D/gi, '')))];
video_details.id = data.videoDetails.videoId;
video_details.title = data.videoDetails.title;
video_details.description = data.videoDetails.shortDescription;
video_details.thumbnail = data.videoDetails.thumbnail.thumbnails.slice(-1)[0];
video_details.metadata = metadata;
} else {
const is_dislike_available = data[3].response.contents.singleColumnWatchNextResults.results.results.contents[1].slimVideoMetadataSectionRenderer.contents[1].slimVideoActionBarRenderer.buttons[1].slimMetadataToggleButtonRenderer.button.toggleButtonRenderer.defaultText.accessibility && true || false;
metadata.embed = data[2].playerResponse.microformat.playerMicroformatRenderer.embed;
metadata.likes = parseInt(data[3].response.contents.singleColumnWatchNextResults.results.results.contents[1].slimVideoMetadataSectionRenderer.contents[1].slimVideoActionBarRenderer.buttons[0].slimMetadataToggleButtonRenderer.button.toggleButtonRenderer.defaultText.accessibility.accessibilityData.label.replace(/\D/g, ''));
metadata.dislikes = is_dislike_available && parseInt(data[3].response.contents.singleColumnWatchNextResults.results.results.contents[1].slimVideoMetadataSectionRenderer.contents[1].slimVideoActionBarRenderer.buttons[1].slimMetadataToggleButtonRenderer.button.toggleButtonRenderer.defaultText.accessibility.accessibilityData.label.replace(/\D/g, '')) || 0;
metadata.view_count = parseInt(data[2].playerResponse.videoDetails.viewCount);
metadata.average_rating = data[2].playerResponse.videoDetails.averageRating;
metadata.length_seconds = data[2].playerResponse.microformat.playerMicroformatRenderer.lengthSeconds;
metadata.channel_id = data[2].playerResponse.videoDetails.channelId;
metadata.channel_url = data[2].playerResponse.microformat.playerMicroformatRenderer.ownerProfileUrl;
metadata.external_channel_id = data[2].playerResponse.microformat.playerMicroformatRenderer.externalChannelId;
metadata.is_live_content = data[2].playerResponse.videoDetails.isLiveContent;
metadata.is_family_safe = data[2].playerResponse.microformat.playerMicroformatRenderer.isFamilySafe;
metadata.is_unlisted = data[2].playerResponse.microformat.playerMicroformatRenderer.isUnlisted;
metadata.is_private = data[2].playerResponse.videoDetails.isPrivate;
metadata.has_ypc_metadata = data[2].playerResponse.microformat.playerMicroformatRenderer.hasYpcMetadata;
metadata.category = data[2].playerResponse.microformat.playerMicroformatRenderer.category;
metadata.channel_name = data[2].playerResponse.microformat.playerMicroformatRenderer.ownerChannelName;
metadata.publish_date = data[2].playerResponse.microformat.playerMicroformatRenderer.publishDate;
metadata.upload_date = data[2].playerResponse.microformat.playerMicroformatRenderer.uploadDate;
metadata.keywords = data[2].playerResponse.videoDetails.keywords;
metadata.available_qualities = [...new Set(data[2].playerResponse.streamingData.adaptiveFormats.filter(v => v.qualityLabel).map(v => v.qualityLabel).sort((a, b) => +a.replace(/\D/gi, '') - +b.replace(/\D/gi, '')))];
video_details.id = data[2].playerResponse.videoDetails.videoId;
video_details.title = data[2].playerResponse.videoDetails.title;
video_details.description = data[2].playerResponse.videoDetails.shortDescription;
video_details.thumbnail = data[2].playerResponse.videoDetails.thumbnail.thumbnails.slice(-1)[0];
// Placeholders for functions
video_details.like = () => {};
video_details.dislike = () => {};
video_details.removeLike = () => {};
video_details.subscribe = () => {};
video_details.unsubscribe = () => {};
video_details.comment = () => {};
video_details.getComments = () => {};
video_details.setNotificationPref = () => {};
video_details.getLivechat = () => {};
// Additional metadata
video_details.metadata = metadata;
}
return video_details;
.replace(/\[function\(d,e,f\)/g, '["function(d,e,f)').replace(/,b,/g, ',"b",')
.replace(/,b/g, ',"b"').replace(/b,/g, '"b",').replace(/b]/g, '"b"]')
.replace(/\[b/g, '["b"').replace(/}]/g, '"]').replace(/},/g, '}",')
.replace(/""/g, '').replace(/length]\)}"/g, 'length])}');
}
};

View File

@@ -5,39 +5,41 @@ const Stream = require('stream');
const OAuth = require('./OAuth');
const Utils = require('./Utils');
const Player = require('./Player');
const Parser = require('./Parser');
const NToken = require('./NToken');
const Actions = require('./Actions');
const Livechat = require('./Livechat');
const Constants = require('./Constants');
const SigDecipher = require('./Sig');
const EventEmitter = require('events');
const TimeToSeconds = require('time-to-seconds');
const CancelToken = Axios.CancelToken;
class Innertube {
constructor(cookie) {
this.cookie = cookie || '';
this.retry_count = 0;
return this.init();
return this.#init();
}
async init() {
async #init() {
const response = await Axios.get(Constants.URLS.YT_BASE_URL, Constants.DEFAULT_HEADERS(this)).catch((error) => error);
if (response instanceof Error) throw new Error(`Could not extract Innertube data: ${response.message}`);
if (response instanceof Error) throw new Error(`Could not retrieve Innertube session: ${response.message}`);
try {
const innertube_data = JSON.parse(`{${Utils.getStringBetweenStrings(response.data, 'ytcfg.set({', '});')}}`);
if (innertube_data.INNERTUBE_CONTEXT) {
this.context = innertube_data.INNERTUBE_CONTEXT;
this.key = innertube_data.INNERTUBE_API_KEY;
this.id_token = innertube_data.ID_TOKEN;
this.session_token = innertube_data.XSRF_TOKEN;
this.player_url = innertube_data.PLAYER_JS_URL;
this.logged_in = innertube_data.LOGGED_IN;
this.sts = innertube_data.STS;
const data = JSON.parse(`{${Utils.getStringBetweenStrings(response.data, 'ytcfg.set({', '});')}}`);
if (data.INNERTUBE_CONTEXT) {
this.context = data.INNERTUBE_CONTEXT;
this.key = data.INNERTUBE_API_KEY;
this.id_token = data.ID_TOKEN;
this.session_token = data.XSRF_TOKEN;
this.player_url = data.PLAYER_JS_URL;
this.logged_in = data.LOGGED_IN;
this.sts = data.STS;
this.context.client.hl = 'en';
this.context.client.gl = 'US';
this.ev = new EventEmitter();
this.player = new Player(this);
await this.player.init();
@@ -45,36 +47,32 @@ class Innertube {
this.auth_apisid = Utils.getStringBetweenStrings(this.cookie, 'PAPISID=', ';');
this.auth_apisid = Utils.generateSidAuth(this.auth_apisid);
}
this.ev = new EventEmitter();
} else {
this.retry_count += 1;
if (this.retry_count >= 10) throw new Error('Could not retrieve Innertube data');
return this.init();
throw new Error('Could not retrieve Innertube session due to unknown reasons');
}
} catch (err) {
this.retry_count += 1;
if (this.retry_count >= 10) throw new Error('Could not retrieve Innertube data');
return this.init();
if (this.retry_count >= 10) throw new Error(`Could not retrieve Innertube session: ${err.message}`);
return this.#init();
}
return this;
}
/**
* Signs-in to a google account.
*
* @param {object} auth_info { refresh_token: string, access_token: string, expires: string }
* @returns {Promise<void>}
*/
signIn(auth_info = {}) {
return new Promise(async (resolve, reject) => {
const oauth = new OAuth(auth_info);
if (auth_info.access_token) {
const is_valid = await oauth.isTokenValid(auth_info.expires);
if (!is_valid) {
const new_tokens = await oauth.refreshAccessToken(auth_info.refresh_token);
auth_info.refresh_token = new_tokens.credentials.refresh_token;
auth_info.access_token = new_tokens.credentials.access_token;
this.ev.emit('update-credentials', {
credentials: new_tokens.credentials,
status: new_tokens.status
});
if (!oauth.isTokenValid()) {
const tokens = await oauth.refreshAccessToken();
auth_info.refresh_token = tokens.credentials.refresh_token;
auth_info.access_token = tokens.credentials.access_token;
this.ev.emit('update-credentials', { credentials: tokens.credentials, status: tokens.status });
}
this.access_token = auth_info.access_token;
@@ -85,15 +83,10 @@ class Innertube {
} else {
oauth.on('auth', (data) => {
if (data.status === 'SUCCESS') {
this.ev.emit('auth', { credentials: data.credentials, status: data.status });
this.access_token = data.credentials.access_token;
this.refresh_token = data.credentials.refresh_token;
this.logged_in = true;
this.ev.emit('auth', {
credentials: data.credentials,
status: data.status
});
resolve();
} else {
this.ev.emit('auth', data);
@@ -103,74 +96,66 @@ class Innertube {
});
}
async search(query, options = { period: 'any', order: 'relevance', duration: 'any' }) {
const response = await Actions.search(this, { query, options });
/**
* Searches on YouTube.
*
* @param {string} query Search query.
* @param {object} options { client: YOUTUBE | YTMUSIC, period: any | hour | day | week | month | year , order: relevance | rating | age | views, duration: any | short | long }
* @returns {Promise<object>} Search results.
*/
async search(query, options = { client: 'YOUTUBE', period: 'any', order: 'relevance', duration: 'any' }) {
const response = await Actions.search(this, options.client, { query, options });
if (!response.success) throw new Error(`Could not search on YouTube: ${response.message}`);
const content = response.data.contents.twoColumnSearchResultsRenderer.primaryContents.sectionListRenderer.contents[0].itemSectionRenderer.contents;
const search = {};
const refined_data = new Parser(this, response.data, {
client: options.client,
data_type: 'SEARCH',
query
}).parse();
search.search_metadata = {};
search.search_metadata.query = content[0].showingResultsForRenderer ? content[0].showingResultsForRenderer.originalQuery.simpleText : query;
search.search_metadata.corrected_query = content[0].showingResultsForRenderer ? content[0].showingResultsForRenderer.correctedQueryEndpoint.searchEndpoint.query : query;
search.search_metadata.estimated_results = parseInt(response.data.estimatedResults);
search.videos = content.map((data) => {
if (!data.videoRenderer) return;
const video = data.videoRenderer;
return {
title: video.title.runs[0].text,
description: video.detailedMetadataSnippets && video.detailedMetadataSnippets[0].snippetText.runs.map((item) => item.text).join('') || 'N/A',
author: video.ownerText.runs[0].text,
id: video.videoId,
url: `https://youtu.be/${video.videoId}`,
channel_url: `${Constants.URLS.YT_BASE_URL}${video.ownerText.runs[0].navigationEndpoint.commandMetadata.webCommandMetadata.url}`,
metadata: {
view_count: video.viewCountText && video.viewCountText.simpleText || 'N/A',
short_view_count_text: {
simple_text: video.shortViewCountText && video.shortViewCountText.simpleText || 'N/A',
accessibility_label: video.shortViewCountText && (video.shortViewCountText.accessibility && video.shortViewCountText.accessibility.accessibilityData.label || 'N/A') || 'N/A',
},
thumbnails: video.thumbnail.thumbnails,
duration: {
seconds: TimeToSeconds(video.lengthText && video.lengthText.simpleText || '0'),
simple_text: video.lengthText && video.lengthText.simpleText || 'N/A',
accessibility_label: video.lengthText && video.lengthText.accessibility.accessibilityData.label || 'N/A'
},
published: video.publishedTimeText && video.publishedTimeText.simpleText || 'N/A',
badges: video.badges && video.badges.map((item) => item.metadataBadgeRenderer.label) || 'N/A',
owner_badges: video.ownerBadges && video.ownerBadges.map((item) => item.metadataBadgeRenderer.tooltip) || 'N/A '
}
};
}).filter((video_block) => video_block !== undefined);
return search;
return refined_data;
}
/**
* Gets details for a video.
*
* @param {string} id The id of the video.
*/
async getDetails(id) {
if (!id) throw new Error('You must provide a video id');
const data = await Actions.getVideoInfo(this, { id, is_desktop: false });
const video_data = Constants.formatVideoData(data, this, false);
const refined_data = new Parser(this, data, { client: 'YOUTUBE', data_type: 'VIDEO_INFO', desktop_v: false }).parse();
if (video_data.metadata.is_live_content) {
if (refined_data.metadata.is_live_content) {
const data_continuation = await Actions.getContinuation(this, { video_id: id });
if (!data_continuation.data.contents.twoColumnWatchNextResults.conversationBar) return;
video_data.getLivechat = () => new Livechat(this, data_continuation.data.contents.twoColumnWatchNextResults.conversationBar.liveChatRenderer.continuations[0].reloadContinuationData.continuation, video_data.metadata.channel_id, id);
if (data_continuation.data.contents.twoColumnWatchNextResults.conversationBar) {
refined_data.getLivechat = () => new Livechat(this, data_continuation.data.contents.twoColumnWatchNextResults.conversationBar.liveChatRenderer.continuations[0].reloadContinuationData.continuation, refined_data.metadata.channel_id, id);
} else {
refined_data.getLivechat = () => { };
}
} else {
video_data.getLivechat = () => {};
refined_data.getLivechat = () => { };
}
video_data.like = () => Actions.engage(this, 'like/like', { video_id: id });
video_data.dislike = () => Actions.engage(this, 'like/dislike', { video_id: id });
video_data.removeLike = () => Actions.engage(this, 'like/removelike', { video_id: id });
video_data.subscribe = () => Actions.engage(this, 'subscription/subscribe', { video_id: id, channel_id: video_data.metadata.channel_id });
video_data.unsubscribe = () => Actions.engage(this, 'subscription/unsubscribe', { video_id: id, channel_id: video_data.metadata.channel_id });
video_data.comment = text => Actions.engage(this, 'comment/create_comment', { video_id: id, text });
video_data.getComments = () => this.getComments(id);
video_data.setNotificationPref = pref => Actions.notifications(this, 'modify_channel_preference', { channel_id: video_data.metadata.channel_id, pref: pref || 'NONE' });
refined_data.like = () => Actions.engage(this, 'like/like', { video_id: id });
refined_data.dislike = () => Actions.engage(this, 'like/dislike', { video_id: id });
refined_data.removeLike = () => Actions.engage(this, 'like/removelike', { video_id: id });
refined_data.subscribe = () => Actions.engage(this, 'subscription/subscribe', { video_id: id, channel_id: refined_data.metadata.channel_id });
refined_data.unsubscribe = () => Actions.engage(this, 'subscription/unsubscribe', { video_id: id, channel_id: refined_data.metadata.channel_id });
refined_data.comment = text => Actions.engage(this, 'comment/create_comment', { video_id: id, text });
refined_data.getComments = () => this.getComments(id);
refined_data.setNotificationPref = pref => Actions.notifications(this, 'modify_channel_preference', { channel_id: refined_data.metadata.channel_id, pref: pref || 'NONE' });
return video_data;
return refined_data;
}
/**
* Gets the comments section of a video.
*
* @param {string} video_id The id of the video.
* @param {string} token Continuation token (optional).
*/
async getComments(video_id, token) {
let comment_section_token;
@@ -220,9 +205,43 @@ class Innertube {
return comments_section;
}
/**
* Returns YouTube's home feed.
* @returns {Promise<object>} home feed.
*/
async getHomeFeed() {
const response = await Actions.browse(this, 'home_feed');
if (!response.success) throw new Error('Could not get home feed');
const contents = response.data.contents.twoColumnBrowseResultsRenderer.tabs[0].tabRenderer.content.richGridRenderer.contents;
return contents.map((item) => {
const content = item.richItemRenderer && item.richItemRenderer.content.videoRenderer &&
item.richItemRenderer.content || undefined;
if (content) return {
id: content.videoRenderer.videoId,
title: content.videoRenderer.title.runs.map((run) => run.text).join(' '),
channel: content.videoRenderer.shortBylineText && content.videoRenderer.shortBylineText.runs[0].text || 'N/A',
metadata: {
view_count: content.videoRenderer.viewCountText && content.videoRenderer.viewCountText.simpleText || 'N/A',
thumbnail: content.videoRenderer.thumbnail && content.videoRenderer.thumbnail.thumbnails.slice(-1)[0] || [],
moving_thumbnail: content.videoRenderer.richThumbnail && content.videoRenderer.richThumbnail.movingThumbnailRenderer.movingThumbnailDetails.thumbnails[0] || [],
published: content.videoRenderer.publishedTimeText && content.videoRenderer.publishedTimeText.simpleText || 'N/A',
badges: content.videoRenderer.badges && content.videoRenderer.badges.map((badge) => badge.metadataBadgeRenderer.label) || 'N/A',
owner_badges: content.videoRenderer.ownerBadges && content.videoRenderer.ownerBadges.map((badge) => badge.metadataBadgeRenderer.tooltip) || 'N/A'
}
}
}).filter((video) => video);
}
/**
* Returns your subscription feed.
* @returns {Promise<object>} subs feed.
*/
async getSubscriptionsFeed() {
const response = await Actions.browse(this, 'subscriptions_feed');
if (!response.success) throw new Error('Could not fetch subscriptions feed');
if (!response.success) throw new Error('Could not get subscriptions feed');
const contents = response.data.contents.twoColumnBrowseResultsRenderer.tabs[0].tabRenderer.content.sectionListRenderer.contents;
const subscriptions_feed = {};
@@ -238,12 +257,12 @@ class Innertube {
section_items.forEach((item) => {
const content = {
title: item.gridVideoRenderer.title.runs.map((run) => run.text).join(' '),
id: item.gridVideoRenderer.videoId,
title: item.gridVideoRenderer.title.runs.map((run) => run.text).join(' '),
channel: item.gridVideoRenderer.shortBylineText && item.gridVideoRenderer.shortBylineText.runs[0].text || 'N/A',
metadata: {
view_count: item.gridVideoRenderer.viewCountText && item.gridVideoRenderer.viewCountText.simpleText || 'N/A',
thumbnail: item.gridVideoRenderer.thumbnail && item.gridVideoRenderer.thumbnail.thumbnails || [],
thumbnail: item.gridVideoRenderer.thumbnail && item.gridVideoRenderer.thumbnail.thumbnails.slice(-1)[0] || [],
published: item.gridVideoRenderer.publishedTimeText && item.gridVideoRenderer.publishedTimeText.simpleText || 'N/A',
badges: item.gridVideoRenderer.badges && item.gridVideoRenderer.badges.map((badge) => badge.metadataBadgeRenderer.label) || 'N/A',
owner_badges: item.gridVideoRenderer.ownerBadges && item.gridVideoRenderer.ownerBadges.map((badge) => badge.metadataBadgeRenderer.tooltip) || 'N/A'
@@ -257,6 +276,10 @@ class Innertube {
return subscriptions_feed;
}
/**
* Returns your notifications.
* @returns {Promise<object>} notifications.
*/
async getNotifications() {
const response = await Actions.notifications(this, 'get_notification_menu');
if (!response.success) throw new Error('Could not fetch notifications');
@@ -279,12 +302,22 @@ class Innertube {
}).filter((notification_block) => notification_block);
}
/**
* Returns the amount of notifications you haven't seen.
* @returns {Promise<number>} unseen notifications count.
*/
async getUnseenNotificationsCount() {
const response = await Actions.notifications(this, 'get_unseen_count');
if (!response.success) throw new Error('Could not fetch unseen notifications count');
if (!response.success) throw new Error('Could not get unseen notifications count');
return response.data.unseenCount;
}
/**
* Downloads a video from YouTube.
*
* @param {string} id The id of the video.
* @param {object} options Download options: { quality?: string, type?: string, format?: string }
*/
download(id, options = {}) {
if (!id) throw new Error('Missing video id');
@@ -326,8 +359,6 @@ class Innertube {
formats.hls_manifest_url = video_data.streamingData.hlsManifestUrl || undefined;
formats.dash_manifest_url = video_data.streamingData.dashManifestUrl || undefined;
const video_details = Constants.formatVideoData(video_data, this, true);
let url;
let bitrates;
let filtered_streams;
@@ -363,7 +394,8 @@ class Innertube {
if (!selected_format) {
return stream.emit('error', { message: 'Could not find any suitable format.', type: 'FORMAT_UNAVAILABLE' });
} else {
stream.emit('info', { video_details, selected_format, formats });
const refined_data = new Parser(this, video_data, { client: 'YOUTUBE', data_type: 'VIDEO_INFO', desktop_v: true }).parse();
stream.emit('info', { video_details: refined_data, selected_format, formats });
}
if (options.type == 'videoandaudio' && !options.range) {
@@ -390,8 +422,8 @@ class Innertube {
});
response.data.on('error', (err) => {
cancelled && stream.emit('error', { message: 'The download was cancelled.', type: 'DOWNLOAD_CANCELLED' }) ||
stream.emit('error', { message: err.message, type: 'DOWNLOAD_ABORTED' });
cancelled && stream.emit('error', { message: 'The download was cancelled.', type: 'DOWNLOAD_CANCELLED' })
|| stream.emit('error', { message: err.message, type: 'DOWNLOAD_ABORTED' });
});
response.data.pipe(stream, { end: true });

View File

@@ -19,10 +19,53 @@ class Livechat extends EventEmitter {
this.poll_intervals_ms = 1000;
this.running = true;
this.poll();
this.#poll();
}
enqueueActionGroup(group) {
async #poll() {
if (!this.running) return;
const livechat = await Actions.livechat(this.session, 'live_chat/get_live_chat', { ctoken: this.ctoken });
if (!livechat.success) {
this.emit('error', { message: `Failed polling livechat: ${livechat.message}. Retrying...` });
return await this.#poll();
}
const continuation_contents = livechat.data.continuationContents;
const action_group = continuation_contents.liveChatContinuation.actions;
this.#enqueueActionGroup(action_group);
this.message_queue.forEach((message) => {
if (this.id_cache.includes(message.id)) return;
setTimeout(() => this.emit('chat-update', message), message.timestamp / 1000 - new Date().getTime());
this.id_cache.push(message.id);
});
this.message_queue = [];
const data = { video_id: this.video_id };
if (this.metadata_ctoken) data.continuation = this.metadata_ctoken;
const updated_metadata = await Actions.livechat(this.session, 'updated_metadata', data);
if (!updated_metadata.success) {
this.emit('error', { message: `Failed polling livechat metadata: ${livechat.message}.` });
}
this.metadata_ctoken = updated_metadata.data.continuation.timedContinuationData.continuation;
const metadata = updated_metadata.data.actions;
this.emit('update-metadata', {
likes: metadata[1].updateToggleButtonTextAction.defaultText.simpleText,
view_count: {
simple_text: metadata[0].updateViewershipAction.viewCount.videoViewCountRenderer.viewCount.simpleText,
short_view_count: metadata[0].updateViewershipAction.viewCount.videoViewCountRenderer.extraShortViewCount.simpleText
}
});
this.livechat_poller = setTimeout(async () => await this.#poll(), this.poll_intervals_ms);
}
#enqueueActionGroup(group) {
group.forEach((action) => {
if (!action.addChatItemAction) return; //TODO: handle different action types
const message_content = action.addChatItemAction.item.liveChatTextMessageRenderer;
@@ -43,48 +86,6 @@ class Livechat extends EventEmitter {
});
}
async poll() {
if (!this.running) return;
let data;
data = { context: this.session.context, continuation: this.ctoken };
const livechat = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/live_chat/get_live_chat${this.session.logged_in && this.session.cookie.length < 1 ? '' : `?key=${this.session.key}`}`, JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session: this.session, data, desktop: true }));
if (livechat instanceof Error) throw new Error(`Error polling livechat: ${livechat.message}`);
const continuation_contents = livechat.data.continuationContents;
const action_group = continuation_contents.liveChatContinuation.actions;
this.enqueueActionGroup(action_group);
// Why don't we just emit the message directly? Well, enqueueing the messages is necessary so they are not emitted in a “messy” way, funny enough that's exactly how YouTube does it in its live chat js script.
this.message_queue.forEach((message, index) => {
if (this.id_cache.includes(message.id)) return;
setTimeout(() => this.emit('chat-update', message), message.timestamp / 1000 - new Date().getTime());
this.id_cache.push(message.id);
});
this.message_queue = [];
data = { context: this.session.context, videoId: this.video_id };
if (this.metadata_ctoken) data.continuation = this.metadata_ctoken;
const updated_metadata = await Axios.post(`${Constants.URLS.YT_BASE_URL}/youtubei/v1/updated_metadata${this.session.logged_in && this.session.cookie.length < 1 ? '' : `?key=${this.session.key}`}`, JSON.stringify(data), Constants.INNERTUBE_REQOPTS({ session: this.session, data, desktop: true }));
if (updated_metadata instanceof Error) throw new Error(`Error polling updated metadata: ${updated_metadata.message}`);
this.metadata_ctoken = updated_metadata.data.continuation.timedContinuationData.continuation;
const metadata = updated_metadata.data.actions;
this.emit('update-metadata', {
likes: metadata[1].updateToggleButtonTextAction.defaultText.simpleText,
dislikes: metadata[2].updateToggleButtonTextAction.defaultText.simpleText,
view_count: {
simple_text: metadata[0].updateViewershipAction.viewCount.videoViewCountRenderer.viewCount.simpleText,
short_view_count: metadata[0].updateViewershipAction.viewCount.videoViewCountRenderer.extraShortViewCount.simpleText
}
});
this.livechat_poller = setTimeout(async () => await this.poll(), this.poll_intervals_ms);
}
async sendMessage(text) {
const message = await Actions.livechat(this.session, 'live_chat/send_message', { text, channel_id: this.channel_id, video_id: this.video_id });
if (!message.success) return message;
@@ -104,7 +105,7 @@ class Livechat extends EventEmitter {
return {
success: true,
status_code: message.status_code,
deleteMessage,
deleteMessage: deleteMessage,
message_data: {
text: message.data.actions[0].addChatItemAction.item.liveChatTextMessageRenderer.message.runs.map((item) => item.text).join(' '),
author: {

View File

@@ -10,100 +10,126 @@ class NToken {
this.transformation_calls_regex = /c\[(.*?)\]\((.+?)\)/g;
}
/**
* Solves throttling challange by transforming the n token.
*
* @param {string} n token.
* @returns {string} transformed token.
*/
transform(n) {
let n_token = n.split('');
try {
let transformations = this.getTransformationData();
let transformations = this.#getTransformationData();
transformations = transformations.map((el) => {
if (el != null && typeof el != 'number') {
const is_reverse_base64 = el.includes('case 65:');
(({ // Identifies the transformation functions and emulates them accordingly.
[Constants.FUNCS.PUSH]: () => el = (arr, i) => this.push(arr, i),
[Constants.FUNCS.SPLICE]: () => el = (arr, i) => this.splice(arr, i),
[Constants.FUNCS.SWAP0_1]: () => el = (arr, i) => this.swap0(arr, i),
[Constants.FUNCS.SWAP0_2]: () => el = (arr, i) => this.swap0(arr, i),
[Constants.FUNCS.ROTATE_1]: () => el = (arr, i) => this.rotate(arr, i),
[Constants.FUNCS.ROTATE_2]: () => el = (arr, i) => this.rotate(arr, i),
[Constants.FUNCS.REVERSE_1]: () => el = (arr) => this.reverse(arr),
[Constants.FUNCS.REVERSE_2]: () => el = (arr) => this.reverse(arr),
[Constants.FUNCS.BASE64_DIA]: () => el = () => this.getBase64Dia(is_reverse_base64),
[Constants.FUNCS.TRANSLATE_1]: () => el = (arr, token) => this.translate1(arr, token, is_reverse_base64),
[Constants.FUNCS.TRANSLATE_2]: () => el = (arr, token, base64_dic) => this.translate2(arr, token, base64_dic)
})[this.getFunc(el)] || (() => el === 'b' && (el = n_token)))();
(({ // Identifies the transformation functions
[Constants.FUNCS.PUSH]: () => el = (arr, i) => this.#push(arr, i),
[Constants.FUNCS.SPLICE]: () => el = (arr, i) => this.#splice(arr, i),
[Constants.FUNCS.SWAP0_1]: () => el = (arr, i) => this.#swap0(arr, i),
[Constants.FUNCS.SWAP0_2]: () => el = (arr, i) => this.#swap0(arr, i),
[Constants.FUNCS.ROTATE_1]: () => el = (arr, i) => this.#rotate(arr, i),
[Constants.FUNCS.ROTATE_2]: () => el = (arr, i) => this.#rotate(arr, i),
[Constants.FUNCS.REVERSE_1]: () => el = (arr) => this.#reverse(arr),
[Constants.FUNCS.REVERSE_2]: () => el = (arr) => this.#reverse(arr),
[Constants.FUNCS.BASE64_DIA]: () => el = () => this.#getBase64Dia(is_reverse_base64),
[Constants.FUNCS.TRANSLATE_1]: () => el = (arr, token) => this.#translate1(arr, token, is_reverse_base64),
[Constants.FUNCS.TRANSLATE_2]: () => el = (arr, token, base64_dic) => this.#translate2(arr, token, base64_dic)
})[this.#getFunc(el)] || (() => el === 'b' && (el = n_token)))();
}
return el;
});
// Fills the null placeholders with a copy of the transformations array.
let null_placeholder_positions = [...this.raw_code.matchAll(this.null_placeholder_regex)].map((item) => parseInt(item[1]));
// Fills the null placeholders with a copy of the transformations array
const null_placeholder_positions = [...this.raw_code.matchAll(this.null_placeholder_regex)].map((item) => parseInt(item[1]));
null_placeholder_positions.forEach((pos) => transformations[pos] = transformations);
// Parses and emulates calls to functions of the transformations array.
let transformation_calls = [...Utils.getStringBetweenStrings(this.raw_code.replace(/\n/g, ''), 'try{', '}catch').matchAll(this.transformation_calls_regex)].map((params) => ({ index: params[1], params: params[2] }));
// Parses and emulates calls to the functions of the transformations array
const transformation_calls = [...Utils.getStringBetweenStrings(this.raw_code.replace(/\n/g, ''), 'try{', '}catch')
.matchAll(this.transformation_calls_regex)].map((params) => ({ index: params[1], params: params[2] }));
transformation_calls.forEach((data) => {
const param_index = data.params.split(',').map((param) => param.match(/c\[(.*?)\]/)[1]);
const base64_dia = (param_index[2] && transformations[param_index[2]]());
transformations[data.index](transformations[param_index[0]], transformations[param_index[1]], base64_dia);
});
} catch (err) {
console.error('Could not transform n-token, download may be throttled:', err);
console.error(`Could not transform n-token (${n}), download may be throttled:`, err)
return n;
}
return n_token.join('');
}
getFunc(el) {
#getFunc(el) {
return el.match(Constants.FUNCS_REGEX);
}
getTransformationData() {
/**
* Takes the n-transform data, refines it, and then returns a readable json array.
* @returns {object}
*/
#getTransformationData() {
const data = `[${Utils.getStringBetweenStrings(this.raw_code.replace(/\n/g, ''), 'c=[', '];c')}]`;
return JSON.parse(Constants.formatNTransformData(data));
}
translate1(arr, token, is_reverse_base64) {
/**
* Gets a base64 alphabet and uses it as a lookup table to modify n.
*/
#translate1(arr, token, is_reverse_base64) {
const characters = is_reverse_base64 && Constants.BASE64_DIALECT.REVERSE || Constants.BASE64_DIALECT.NORMAL;
arr.forEach(function(char, index, loc) {
arr.forEach(function (char, index, loc) {
this.push(loc[index] = characters[(characters.indexOf(char) - characters.indexOf(this[index]) + 64) % characters.length]);
}, token.split(''));
}
translate2(arr, token, characters) {
#translate2(arr, token, characters) {
let chars_length = characters.length;
arr.forEach(function(char, index, loc) {
arr.forEach(function (char, index, loc) {
this.push(loc[index] = characters[(characters.indexOf(char) - characters.indexOf(this[index]) + index + chars_length--) % characters.length]);
}, token.split(''));
}
getBase64Dia(is_reverse_base64) {
/**
* Returns the requested base64 dialect, currently this is only used by 'translate2'.
*/
#getBase64Dia(is_reverse_base64) {
const characters = is_reverse_base64 && Constants.BASE64_DIALECT.REVERSE || Constants.BASE64_DIALECT.NORMAL;
return characters;
}
swap0(arr, index) {
const old_value = arr[0];
/**
* Swaps the first element with the one at the given index.
*/
#swap0(arr, index) {
const old_elem = arr[0];
index = (index % arr.length + arr.length) % arr.length;
arr[0] = arr[index];
arr[index] = old_value;
arr[index] = old_elem;
}
rotate(arr, index) {
/**
* Rotates elements of the array.
*/
#rotate(arr, index) {
index = (index % arr.length + arr.length) % arr.length;
arr.splice(-index).reverse().forEach((el) => arr.unshift(el));
}
splice(arr, index) {
/**
* Deletes one element at the given index.
*/
#splice(arr, index) {
index = (index % arr.length + arr.length) % arr.length;
arr.splice(index, 1);
}
reverse(arr) {
#reverse(arr) {
arr.reverse();
}
push(arr, item) {
#push(arr, item) {
arr.push(item);
}
}

View File

@@ -1,7 +1,6 @@
'use strict';
const Axios = require('axios');
const Utils = require('./Utils');
const Constants = require('./Constants');
const EventEmitter = require('events');
const Uuid = require('uuid');
@@ -9,6 +8,7 @@ const Uuid = require('uuid');
class OAuth extends EventEmitter {
constructor(auth_info) {
super();
this.auth_info = auth_info;
this.refresh_interval = 5;
this.oauth_code_url = `${Constants.URLS.YT_BASE_URL}/o/oauth2/device/code`;
@@ -20,14 +20,17 @@ class OAuth extends EventEmitter {
this.scope = Constants.OAUTH.SCOPE;
this.auth_script_regex = /<script id=\"base-js\" src=\"(.*?)\" nonce=".*?"><\/script>/;
this.identity_regex = /var .+?=\"(?<id>.+?)\",.+?=\"(?<secret>.+?)\"/;
this.identity_regex = /.+?={};var .+?={clientId:\"(?<id>.+?)\",si:\"(?<secret>.+?)\"},/;
if (auth_info.access_token) return;
this.requestAuthCode();
this.#requestAuthCode();
}
async requestAuthCode() {
const identity = await this.getClientIdentity();
/**
* Asks the OAuth server for an auth code.
*/
async #requestAuthCode() {
const identity = await this.#getClientIdentity();
this.client_id = identity.id;
this.client_secret = identity.secret;
@@ -56,11 +59,15 @@ class OAuth extends EventEmitter {
this.refresh_interval = response.data.interval;
// Keeps requesting at a specific rate until the authorization is granted or denied.
this.waitForAuth(response.data.device_code);
this.#waitForAuth(response.data.device_code);
}
waitForAuth(device_code) {
/**
* Waits for sign-in authorization.
*
* @param {string} device_code Client's device code.
*/
#waitForAuth(device_code) {
const data = {
client_id: this.client_id,
client_secret: this.client_secret,
@@ -80,7 +87,7 @@ class OAuth extends EventEmitter {
switch (response.data.error) {
case 'slow_down':
case 'authorization_pending':
this.waitForAuth(device_code);
this.#waitForAuth(device_code);
break;
case 'access_denied':
this.emit('auth', {
@@ -93,7 +100,7 @@ class OAuth extends EventEmitter {
error: 'The device code has expired, requesting a new one.',
status: 'DEVICE_CODE_EXPIRED'
});
this.requestAuthCode();
this.#requestAuthCode();
break;
default:
}
@@ -113,13 +120,17 @@ class OAuth extends EventEmitter {
}, 1000 * this.refresh_interval);
}
async refreshAccessToken(refresh_token) {
const identity = await this.getClientIdentity();
/**
* Gets a new access token using a refresh token.
* @returns {object} { credentials: { access_token: string, refresh_token: string, expires: string }, status: 'FAILED' | 'SUCCESS' }
*/
async refreshAccessToken() {
const identity = await this.#getClientIdentity();
const data = {
client_id: identity.id,
client_secret: identity.secret,
refresh_token,
refresh_token: this.auth_info.refresh_token,
grant_type: 'refresh_token',
};
@@ -144,7 +155,7 @@ class OAuth extends EventEmitter {
return {
credentials: {
refresh_token: refresh_token,
refresh_token: this.auth_info.refresh_token,
access_token: response.data.access_token,
expires: expiration_date
},
@@ -153,29 +164,35 @@ class OAuth extends EventEmitter {
};
}
async isTokenValid(expiration_date) {
const timestamp = new Date(expiration_date).getTime();
const is_valid = new Date().getTime() < timestamp;
return is_valid;
}
async getClientIdentity() {
// The first request is made to get the auth script url, hard-coding it isn't viable as it changes overtime.
/**
* Gets client identity data.
* @returns {object} { id: string, secret: string }
*/
async #getClientIdentity() {
// This request is made to get the auth script url, hard-coding it isn't viable as it changes overtime.
const yttv_response = await Axios.get(`${Constants.URLS.YT_BASE_URL}/tv`, Constants.OAUTH.HEADERS).catch((error) => error);
if (yttv_response instanceof Error) throw new Error(`Could not extract client identity: ${yttv_response.message}`);
// Here we get the script and extract the necessary data to proceed with the auth flow.
// Here we download the script and extract the necessary data to proceed with the auth flow.
const url_body = this.auth_script_regex.exec(yttv_response.data)[1];
const script_url = `${Constants.URLS.YT_BASE_URL}/${url_body}`;
const response = await Axios.get(script_url, Constants.DEFAULT_HEADERS).catch((error) => error);
if (response instanceof Error) throw new Error(`Could not extract client identity: ${response.message}`);
const identity_function = Utils.getStringBetweenStrings(response.data, 'setQuery("");', '{useGaiaSandbox:');
const client_identity = identity_function.replace(/\n/g, '').match(this.identity_regex);
const client_identity = response.data.replace(/\n/g, '').match(this.identity_regex);
return client_identity.groups;
}
/**
* Checks access token validity.
* @returns {boolean} true | false
*/
isTokenValid() {
const timestamp = new Date(this.auth_info.expires).getTime();
const is_valid = new Date().getTime() < timestamp;
return is_valid;
}
}
module.exports = OAuth;

206
lib/Parser.js Normal file
View File

@@ -0,0 +1,206 @@
'use strict';
const Utils = require('./Utils');
const Actions = require('./Actions');
const Constants = require('./Constants');
/**
* Takes raw data from the Innertube API and refines it.
* Mainly used for video data and search results, as those are more complex to parse.
*/
class Parser {
constructor(session, data, args = {}) {
this.session = session;
this.data = data;
this.args = args;
}
parse() {
return this.args.client === 'YOUTUBE' ? ({
SEARCH: () => this.#parseVideoSearch(),
VIDEO_INFO: () => this.#parseVideoInfo()
})[this.args.data_type]() : ({
SEARCH: () => this.#parseMusicSearch(),
SONG_INFO: () => { }
})[this.args.data_type]();
}
#parseVideoSearch() {
const response = {};
const contents = this.data.contents.twoColumnSearchResultsRenderer
.primaryContents.sectionListRenderer.contents[0].itemSectionRenderer
.contents;
const continuation_token = this.data.contents.twoColumnSearchResultsRenderer
.primaryContents.sectionListRenderer.contents[1].continuationItemRenderer
.continuationEndpoint.continuationCommand.token;
response.search_metadata = {};
response.search_metadata.query = contents[0].showingResultsForRenderer && contents[0].showingResultsForRenderer.originalQuery.simpleText || this.args.query;
response.search_metadata.corrected_query = contents[0].showingResultsForRenderer && contents[0].showingResultsForRenderer.correctedQueryEndpoint.searchEndpoint.query || this.args.query;
response.search_metadata.estimated_results = parseInt(this.data.estimatedResults);
response.videos = contents.map((data) => {
if (!data.videoRenderer) return;
const video = data.videoRenderer;
return {
title: video.title.runs[0].text,
description: video.detailedMetadataSnippets && video.detailedMetadataSnippets[0].snippetText.runs.map((item) => item.text).join('') || 'N/A',
author: video.ownerText.runs[0].text,
id: video.videoId,
url: `https://youtu.be/${video.videoId}`,
channel_url: `${Constants.URLS.YT_BASE_URL}${video.ownerText.runs[0].navigationEndpoint.commandMetadata.webCommandMetadata.url}`,
metadata: {
view_count: video.viewCountText && video.viewCountText.simpleText || 'N/A',
short_view_count_text: {
simple_text: video.shortViewCountText && video.shortViewCountText.simpleText || 'N/A',
accessibility_label: video.shortViewCountText && (video.shortViewCountText.accessibility && video.shortViewCountText.accessibility.accessibilityData.label || 'N/A') || 'N/A',
},
thumbnails: video.thumbnail.thumbnails,
duration: {
seconds: Utils.timeToSeconds(video.lengthText && video.lengthText.simpleText || '0'),
simple_text: video.lengthText && video.lengthText.simpleText || 'N/A',
accessibility_label: video.lengthText && video.lengthText.accessibility.accessibilityData.label || 'N/A'
},
published: video.publishedTimeText && video.publishedTimeText.simpleText || 'N/A',
badges: video.badges && video.badges.map((item) => item.metadataBadgeRenderer.label) || 'N/A',
owner_badges: video.ownerBadges && video.ownerBadges.map((item) => item.metadataBadgeRenderer.tooltip) || 'N/A '
}
};
}).filter((video) => video);
return response;
}
#parseMusicSearch() {
const tabs = this.data.contents.tabbedSearchResultsRenderer.tabs;
const contents = tabs[0].tabRenderer.content.sectionListRenderer.contents;
/**
* WIP
**/
const getLyrics = async (id) => {
// const data_continuation = await Actions.getContinuation(this.session, { video_id: id, ytmusic: true });
return undefined;
}
const songs_ms = contents.find((content) => content.musicShelfRenderer.title.runs[0].text == 'Songs');
const songs = songs_ms.musicShelfRenderer.contents.map((item) => {
const list_item = item.musicResponsiveListItemRenderer;
return {
id: list_item.playlistItemData.videoId,
title: list_item.flexColumns[0].musicResponsiveListItemFlexColumnRenderer.text.runs[0].text,
artist: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[2].text,
album: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[4].text,
duration: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[6].text,
thumbnail: list_item.thumbnail.musicThumbnailRenderer.thumbnail,
getLyrics: () => getLyrics(list_item.playlistItemData.videoId)
};
});
const videos_ms = contents.find((content) => content.musicShelfRenderer.title.runs[0].text == 'Videos');
const videos = videos_ms.musicShelfRenderer.contents.map((item) => {
const list_item = item.musicResponsiveListItemRenderer;
return {
id: list_item.playlistItemData.videoId,
title: list_item.flexColumns[0].musicResponsiveListItemFlexColumnRenderer.text.runs[0].text,
author: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[2].text,
views: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[4].text,
duration: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[6].text,
thumbnail: list_item.thumbnail.musicThumbnailRenderer.thumbnail,
getLyrics: () => getLyrics(list_item.playlistItemData.videoId)
};
});
const albums_ms = contents.find((content) => content.musicShelfRenderer.title.runs[0].text == 'Albums');
const albums = albums_ms.musicShelfRenderer.contents.map((item) => {
const list_item = item.musicResponsiveListItemRenderer;
return {
title: list_item.flexColumns[0].musicResponsiveListItemFlexColumnRenderer.text.runs[0].text,
author: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs[2].text,
year: list_item.flexColumns[1].musicResponsiveListItemFlexColumnRenderer.text.runs.find((run) => /^[12][0-9]{3}$/.test(run.text)).text,
thumbnail: list_item.thumbnail.musicThumbnailRenderer.thumbnail,
};
});
return { songs, videos, albums };
}
#parseVideoInfo() {
const desktop_v = this.args.desktop_v;
const playability_status = desktop_v && this.data.playabilityStatus ||
this.data[2].playerResponse.playabilityStatus;
if (playability_status.status == 'ERROR')
throw new Error(`Could not retrieve details for this video: ${playability_status.status} - ${playability_status.reason}`);
const details = desktop_v && this.data.videoDetails ||
this.data[2].playerResponse.videoDetails;
const microformat = desktop_v && this.data.microformat.playerMicroformatRenderer ||
this.data[2].playerResponse.microformat.playerMicroformatRenderer;
const streaming_data = desktop_v && this.data.streamingData ||
this.data[2].playerResponse.streamingData;
const response = {
id: '',
title: '',
description: '',
thumbnail: [],
metadata: {}
};
const mf_raw_data = Object.entries(microformat);
const dt_raw_data = Object.entries(details);
mf_raw_data.forEach((entry) => {
const key = Utils.camelToSnake(entry[0]);
if (Constants.METADATA_KEYS.includes(key)) {
key == 'view_count' && (response.metadata[key] = parseInt(entry[1])) ||
key == 'owner_profile_url' && (response.metadata.channel_url = entry[1]) ||
key == 'owner_channel_name' && (response.metadata.channel_name = entry[1]) ||
(response.metadata[key] = entry[1]);
} else {
response[key] = entry[1];
}
});
dt_raw_data.forEach((entry) => {
const key = Utils.camelToSnake(entry[0]);
if (Constants.BLACKLISTED_KEYS.includes(key)) return;
if (Constants.METADATA_KEYS.includes(key)) {
key == 'view_count' && (response.metadata[key] = parseInt(entry[1])) ||
(response.metadata[key] = entry[1]);
} else {
key == 'short_description' && (response.description = entry[1]) ||
key == 'thumbnail' && (response.thumbnail = entry[1].thumbnails.slice(-1)[0]) ||
key == 'video_id' && (response.id = entry[1]) ||
(response[key] = entry[1]);
}
});
if (!desktop_v) {
const dislike_available = this.data[3].response.contents.singleColumnWatchNextResults.results.results.contents[1]
.slimVideoMetadataSectionRenderer.contents[1].slimVideoActionBarRenderer.buttons[1].slimMetadataToggleButtonRenderer
.button.toggleButtonRenderer.defaultText.accessibility && true || false;
response.metadata.likes = parseInt(this.data[3].response.contents.singleColumnWatchNextResults.results.results.contents[1]
.slimVideoMetadataSectionRenderer.contents[1].slimVideoActionBarRenderer.buttons[0].slimMetadataToggleButtonRenderer
.button.toggleButtonRenderer.defaultText.accessibility.accessibilityData.label.replace(/\D/g, ''));
response.metadata.dislikes = dislike_available && parseInt(this.data[3].response.contents.singleColumnWatchNextResults.results.results.contents[1]
.slimVideoMetadataSectionRenderer.contents[1].slimVideoActionBarRenderer.buttons[1].slimMetadataToggleButtonRenderer
.button.toggleButtonRenderer.defaultText.accessibility.accessibilityData.label.replace(/\D/g, '')) || 0;
}
response.metadata.available_qualities = [...new Set(streaming_data.adaptiveFormats.filter(v => v.qualityLabel)
.map(v => v.qualityLabel).sort((a, b) => + a.replace(/\D/gi, '') - + b.replace(/\D/gi, '')))];
return response;
}
}
module.exports = Parser;

View File

@@ -1,45 +1,45 @@
'use strict';
const fs = require('fs');
const Fs = require('fs');
const Axios = require('axios');
const Utils = require('./Utils');
const Constants = require('./Constants');
class Player {
constructor(innertube_session) {
this.session = innertube_session;
constructor(session) {
this.session = session;
this.player_name = Utils.getStringBetweenStrings(this.session.player_url, '/player/', '/');
this.tmp_cache_dir = __dirname.slice(0, -3) + 'cache';
}
async init() {
if (fs.existsSync(`${this.tmp_cache_dir}/${this.player_name}.js`)) {
const player_data = fs.readFileSync(`${this.tmp_cache_dir}/${this.player_name}.js`).toString();
this.getSigDecipherCode(player_data);
this.getNEncoder(player_data);
if (Fs.existsSync(`${this.tmp_cache_dir}/${this.player_name}.js`)) {
const player_data = Fs.readFileSync(`${this.tmp_cache_dir}/${this.player_name}.js`).toString();
this.sig_decipher_sc = this.#getSigDecipherCode(player_data);
this.ntoken_sc = this.#getNEncoder(player_data);
} else {
const response = await Axios.get(`${Constants.URLS.YT_BASE_URL}${this.session.player_url}`, { path: this.session.playerUrl, headers: { 'content-type': 'text/javascript', 'user-agent': Utils.getRandomUserAgent('desktop').userAgent } }).catch((error) => error);
if (response instanceof Error) throw new Error('Could not download player script: ' + response.message);
try {
// Caches the current player so we don't have to download it all the time
fs.mkdirSync(this.tmp_cache_dir, { recursive: true });
fs.writeFileSync(`${this.tmp_cache_dir}/${this.player_name}.js`, response.data);
// Caches the current player so we don't have to download it all the time.
Fs.mkdirSync(this.tmp_cache_dir, { recursive: true });
Fs.writeFileSync(`${this.tmp_cache_dir}/${this.player_name}.js`, response.data);
} catch (err) {}
this.getSigDecipherCode(response.data);
this.getNEncoder(response.data);
this.sig_decipher_sc = this.#getSigDecipherCode(response.data);
this.ntoken_sc = this.#getNEncoder(response.data);
}
}
getSigDecipherCode(data) {
const manipulation_algorithm_code = Utils.getStringBetweenStrings(data, 'this.audioTracks};var', '};');
const manipulation_sequence_code = Utils.getStringBetweenStrings(data, 'function(a){a=a.split("")', 'return a.join("")}');
this.sig_decipher_sc = manipulation_algorithm_code + manipulation_sequence_code;
#getSigDecipherCode(data) {
const sig_alg_sc = Utils.getStringBetweenStrings(data, 'this.audioTracks};var', '};');
const sig_data = Utils.getStringBetweenStrings(data, 'function(a){a=a.split("")', 'return a.join("")}');
return sig_alg_sc + sig_data;
}
getNEncoder(data) {
this.ntoken_sc = `var b=a.split("")${Utils.getStringBetweenStrings(data, 'b=a.split("")', '}return b.join("")}')}} return b.join("");`;
#getNEncoder(data) {
return `var b=a.split("")${Utils.getStringBetweenStrings(data, 'b=a.split("")', '}return b.join("")}')}} return b.join("");`;
}
}

View File

@@ -12,9 +12,12 @@ class SigDecipher {
this.actions_regex = /;.{2}\.(.{2})\(.*?,(.*?)\)/g;
}
/**
* Deciphers signature.
*/
decipher() {
const args = QueryString.parse(this.url);
const functions = this.getFunctions();
const functions = this.#getFunctions();
function splice(arr, end) {
arr.splice(0, end);
@@ -56,7 +59,7 @@ class SigDecipher {
return url_components.toString();
}
getFunctions() {
#getFunctions() {
let func;
let func_name = [];

View File

@@ -5,6 +5,11 @@ const Proto = require('protons');
const Crypto = require('crypto');
const UserAgent = require('user-agents');
/**
* Returns a random user agent.
*
* @param {string} type mobile | desktop
*/
function getRandomUserAgent(type) {
switch (type) {
case 'mobile':
@@ -15,6 +20,11 @@ function getRandomUserAgent(type) {
}
}
/**
* Generates an authentication token from a cookies' sid.
*
* @param {string} sid Sid extracted from cookies
*/
function generateSidAuth(sid) {
const youtube = 'https://www.youtube.com';
const timestamp = Math.floor(new Date().getTime() / 1000);
@@ -27,6 +37,14 @@ function generateSidAuth(sid) {
return ['SAPISIDHASH', [timestamp, gen_hash].join('_')].join(' ');
}
/**
* Gets a string between two delimiters.
*
* @param {string} data The data.
* @param {string} start_string Start string.
* @param {string} end_string End string.
*/
function getStringBetweenStrings(data, start_string, end_string) {
const regex = new RegExp(`${escapeStringRegexp(start_string)}(.*?)${escapeStringRegexp(end_string)}`, "s");
const match = data.match(regex);
@@ -37,6 +55,36 @@ function escapeStringRegexp(string) {
return string.replace(/[|\\{}()[\]^$+*?.]/g, "\\$&").replace(/-/g, "\\x2d");
}
/**
* Converts time (h:m:s) to seconds.
*
* @param {string} time
* @returns {string} seconds
*/
function timeToSeconds(time) {
let params = time.split(':');
return parseInt(({
3: +params[0] * 3600 + +params[1] * 60 + +params[2],
2: +params[0] * 60 + +params[1],
1: +params[0]
})[params.length]);
}
/**
* Converts strings in camelCase to snake_case.
*
* @param {string} string The string in camelCase.
*/
function camelToSnake(string) {
return string[0].toLowerCase() + string.slice(1, string.length).replace(/[A-Z]/g, (letter) => `_${letter.toLowerCase()}`);
}
/**
* Encodes notification preferences protobuf.
*
* @param {string} channel_id
* @param {string} index
*/
function encodeNotificationPref(channel_id, index) {
const youtube_proto = Proto(Fs.readFileSync(`${__dirname}/proto/youtube.proto`));
@@ -52,7 +100,14 @@ function encodeNotificationPref(channel_id, index) {
return encodeURIComponent(Buffer.from(buf).toString('base64'));
}
function generateMessageParams(channel_id, video_id) {
/**
* Encodes livestream message protobuf.
*
* @param {string} channel_id
* @param {string} video_id
*/
function encodeMessageParams(channel_id, video_id) {
const youtube_proto = Proto(Fs.readFileSync(`${__dirname}/proto/youtube.proto`));
const buf = youtube_proto.LiveMessageParams.encode({
@@ -69,7 +124,13 @@ function generateMessageParams(channel_id, video_id) {
return Buffer.from(encodeURIComponent(Buffer.from(buf).toString('base64'))).toString('base64');
}
function generateCommentParams(video_id) {
/**
* Encodes comment params protobuf.
*
* @param {string} video_id
*/
function encodeCommentParams(video_id) {
const youtube_proto = Proto(Fs.readFileSync(`${__dirname}/proto/youtube.proto`));
const buf = youtube_proto.CreateCommentParams.encode({
@@ -83,13 +144,21 @@ function generateCommentParams(video_id) {
return encodeURIComponent(Buffer.from(buf).toString('base64'));
}
/**
* Encodes search filter protobuf
*
* @param {string} period Period in which a video is uploaded: any | hour | day | week | month | year
* @param {string} duration The duration of a video: any | short | long
* @param {string} order The order of the search results: relevance | rating | age | views
*/
function encodeFilter(period, duration, order) {
const youtube_proto = Proto(Fs.readFileSync(`${__dirname}/proto/youtube.proto`));
const periods = { 'any': null, 'hour': 1, 'day': 2, 'week': 3, 'month': 4, 'year': 5 };
const durations = { 'any': null, 'short': 1, 'long': 2 };
const orders = { 'relevance': null, 'rating': 1, 'age': 2, 'views': 3 };
const durations = { 'any': null, 'short' : 1, 'long': 2 };
const orders = { 'relevance': null, 'rating': 1, 'age': 2, 'views' : 3 };
const search_filter_buff = youtube_proto.SearchFilter.encode({
number: orders[order],
filter: {
@@ -98,8 +167,8 @@ function encodeFilter(period, duration, order) {
param_2: durations[duration]
}
});
return encodeURIComponent(Buffer.from(search_filter_buff).toString('base64'));
}
module.exports = { getRandomUserAgent, generateSidAuth, getStringBetweenStrings, generateMessageParams, generateCommentParams, encodeNotificationPref, encodeFilter };
module.exports = { getRandomUserAgent, generateSidAuth, getStringBetweenStrings, camelToSnake, timeToSeconds, encodeMessageParams, encodeCommentParams, encodeNotificationPref, encodeFilter };

57
package-lock.json generated
View File

@@ -1,17 +1,16 @@
{
"name": "youtubei.js",
"version": "1.2.8",
"version": "1.2.9",
"lockfileVersion": 2,
"requires": true,
"packages": {
"": {
"name": "youtubei.js",
"version": "1.2.8",
"version": "1.2.9",
"license": "MIT",
"dependencies": {
"axios": "^0.21.4",
"protons": "^2.0.3",
"time-to-seconds": "^1.1.5",
"user-agents": "^1.0.778",
"uuid": "^8.3.2"
}
@@ -54,9 +53,9 @@
}
},
"node_modules/follow-redirects": {
"version": "1.14.6",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.14.6.tgz",
"integrity": "sha512-fhUl5EwSJbbl8AR+uYL2KQDxLkdSjZGR36xy46AO7cOMTrCMON6Sa28FmAnC2tRTDbd/Uuzz3aJBv7EBN7JH8A==",
"version": "1.14.7",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.14.7.tgz",
"integrity": "sha512-+hbxoLbFMbRKDwohX8GkTataGqO6Jb7jGwpAlwgy2bIz25XtRm7KEzJM76R1WiNT5SwZkX4Y75SwBolkpmE7iQ==",
"funding": [
{
"type": "individual",
@@ -78,9 +77,9 @@
"integrity": "sha1-4j8/nE+Pvd6HJSnBBxhXoIblzO8="
},
"node_modules/multiformats": {
"version": "9.5.4",
"resolved": "https://registry.npmjs.org/multiformats/-/multiformats-9.5.4.tgz",
"integrity": "sha512-MFT8e8BOLX7OZKfSBGm13FwYvJVI6MEcZ7hujUCpyJwvYyrC1anul50A0Ee74GdeJ77aYTO6YU1vO+oF8NqSIw=="
"version": "9.6.2",
"resolved": "https://registry.npmjs.org/multiformats/-/multiformats-9.6.2.tgz",
"integrity": "sha512-1dKng7RkBelbEZQQD2zvdzYKgUmtggpWl+GXQBYhnEGGkV6VIYfWgV3VSeyhcUFFEelI5q4D0etCJZ7fbuiamQ=="
},
"node_modules/protocol-buffers-schema": {
"version": "3.6.0",
@@ -106,15 +105,6 @@
"varint": "~5.0.0"
}
},
"node_modules/time-to-seconds": {
"version": "1.1.5",
"resolved": "https://registry.npmjs.org/time-to-seconds/-/time-to-seconds-1.1.5.tgz",
"integrity": "sha512-mpzJDHGF4VdhiahyusCUSy+BWJdN3q8Cluzfy0n7GMU9IIj+HJDX9bbbr7wVSUiqmRn1vqhhfECgdfj+SByu2A==",
"engines": {
"node": ">=15.0.1",
"vscode": "^1.22.0"
}
},
"node_modules/uint8arrays": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/uint8arrays/-/uint8arrays-3.0.0.tgz",
@@ -137,9 +127,9 @@
}
},
"node_modules/user-agents": {
"version": "1.0.869",
"resolved": "https://registry.npmjs.org/user-agents/-/user-agents-1.0.869.tgz",
"integrity": "sha512-fjHZWYMi0CBF8ni/TIcCUFB7GiTTT+UcCAxT9/kNbPd/d5PR9VZ0wOeWZ5McbdJ42PRdFWc1ZZUMJIvpNSsewg==",
"version": "1.0.912",
"resolved": "https://registry.npmjs.org/user-agents/-/user-agents-1.0.912.tgz",
"integrity": "sha512-vvFMlK3XcaF/ZRF1Ky4IrBASSNXflZys+5ArPRanNeggWtOYeD2k3FD/6wrv9h7lFlvh5KS+X45E/siw26+EJg==",
"dependencies": {
"dot-json": "^1.2.2",
"lodash.clonedeep": "^4.5.0"
@@ -189,9 +179,9 @@
}
},
"follow-redirects": {
"version": "1.14.6",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.14.6.tgz",
"integrity": "sha512-fhUl5EwSJbbl8AR+uYL2KQDxLkdSjZGR36xy46AO7cOMTrCMON6Sa28FmAnC2tRTDbd/Uuzz3aJBv7EBN7JH8A=="
"version": "1.14.7",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.14.7.tgz",
"integrity": "sha512-+hbxoLbFMbRKDwohX8GkTataGqO6Jb7jGwpAlwgy2bIz25XtRm7KEzJM76R1WiNT5SwZkX4Y75SwBolkpmE7iQ=="
},
"lodash.clonedeep": {
"version": "4.5.0",
@@ -199,9 +189,9 @@
"integrity": "sha1-4j8/nE+Pvd6HJSnBBxhXoIblzO8="
},
"multiformats": {
"version": "9.5.4",
"resolved": "https://registry.npmjs.org/multiformats/-/multiformats-9.5.4.tgz",
"integrity": "sha512-MFT8e8BOLX7OZKfSBGm13FwYvJVI6MEcZ7hujUCpyJwvYyrC1anul50A0Ee74GdeJ77aYTO6YU1vO+oF8NqSIw=="
"version": "9.6.2",
"resolved": "https://registry.npmjs.org/multiformats/-/multiformats-9.6.2.tgz",
"integrity": "sha512-1dKng7RkBelbEZQQD2zvdzYKgUmtggpWl+GXQBYhnEGGkV6VIYfWgV3VSeyhcUFFEelI5q4D0etCJZ7fbuiamQ=="
},
"protocol-buffers-schema": {
"version": "3.6.0",
@@ -227,11 +217,6 @@
"varint": "~5.0.0"
}
},
"time-to-seconds": {
"version": "1.1.5",
"resolved": "https://registry.npmjs.org/time-to-seconds/-/time-to-seconds-1.1.5.tgz",
"integrity": "sha512-mpzJDHGF4VdhiahyusCUSy+BWJdN3q8Cluzfy0n7GMU9IIj+HJDX9bbbr7wVSUiqmRn1vqhhfECgdfj+SByu2A=="
},
"uint8arrays": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/uint8arrays/-/uint8arrays-3.0.0.tgz",
@@ -254,9 +239,9 @@
}
},
"user-agents": {
"version": "1.0.869",
"resolved": "https://registry.npmjs.org/user-agents/-/user-agents-1.0.869.tgz",
"integrity": "sha512-fjHZWYMi0CBF8ni/TIcCUFB7GiTTT+UcCAxT9/kNbPd/d5PR9VZ0wOeWZ5McbdJ42PRdFWc1ZZUMJIvpNSsewg==",
"version": "1.0.912",
"resolved": "https://registry.npmjs.org/user-agents/-/user-agents-1.0.912.tgz",
"integrity": "sha512-vvFMlK3XcaF/ZRF1Ky4IrBASSNXflZys+5ArPRanNeggWtOYeD2k3FD/6wrv9h7lFlvh5KS+X45E/siw26+EJg==",
"requires": {
"dot-json": "^1.2.2",
"lodash.clonedeep": "^4.5.0"
@@ -273,4 +258,4 @@
"integrity": "sha512-lKxKYG6H03yCZUpAGOPOsMcGxd1RHCu1iKvEHYDPmTyq2HueGhD73ssNBqqQWfvYs04G9iUFRvmAVLW20Jw6ow=="
}
}
}
}

View File

@@ -1,6 +1,6 @@
{
"name": "youtubei.js",
"version": "1.2.8",
"version": "1.2.9",
"description": "An object-oriented library that allows you to search, get detailed info about videos, subscribe, unsubscribe, like, dislike, comment, download videos and much more!",
"main": "index.js",
"scripts": {
@@ -15,7 +15,6 @@
"dependencies": {
"axios": "^0.21.4",
"protons": "^2.0.3",
"time-to-seconds": "^1.1.5",
"user-agents": "^1.0.778",
"uuid": "^8.3.2"
},
@@ -26,6 +25,7 @@
"keywords": [
"youtube",
"youtube-dl",
"youtubedl",
"innertube",
"innertubeapi",
"livechat",
@@ -34,7 +34,6 @@
"like",
"dislike",
"comment",
"automation",
"downloader",
"comments-section",
"youtube-downloader"
@@ -43,4 +42,4 @@
"url": "https://github.com/LuanRT/YouTube.js/issues"
},
"homepage": "https://github.com/LuanRT/YouTube.js#readme"
}
}

View File

@@ -6,8 +6,7 @@ module.exports = {
original_ntoken: 'PqjqqJjdB9K821VIisj',
expected_ntoken: 'AxwyS-osUl1WhMUd1',
client_version: '2.20211101.01.00',
test_video_id: 'FT_nzxtgXEw',
test_video_id_1: 'YE7VzlLtp-4',
test_video_id: 'dQw4w9WgXcQ',
sig_decipher_sc: `fB={RP:function(a,b){a.splice(0,b)},
Td:function(a){a.reverse()},
kq:function(a,b){var c=a[0];a[0]=a[b%a.length];a[b%a.length]=c};fB.kq(a,35);fB.RP(a,2);fB.kq(a,46);fB.Td(a,6);`,

View File

@@ -14,18 +14,19 @@ async function performTests() {
if (!(youtube instanceof Error)) {
const search = await youtube.search('Carl Sagan - Documentary').catch((error) => error);
assert((search instanceof Error ? false : true) && search.videos.length >= 1, `should search videos`, search);
assert(!(search instanceof Error) && search.videos.length >= 1, `should search videos`, search);
const details = await youtube.getDetails(Constants.test_video_id).catch((error) => error);
assert(details instanceof Error ? false : true, `should retrieve details for ${Constants.test_video_id}`, details);
assert(!(details instanceof Error), `should retrieve details for ${Constants.test_video_id}`, details);
const comments = await youtube.getComments(Constants.test_video_id).catch((error) => error);
assert(comments instanceof Error ? false : true, `should retrieve comments for ${Constants.test_video_id}`, comments);
assert(!(comments instanceof Error), `should retrieve comments for ${Constants.test_video_id}`, comments);
const video = await downloadVideo(Constants.test_video_id_1, youtube).catch((error) => error);
assert(video instanceof Error ? false : true, `should download video (${Constants.test_video_id_1})`, video);
const video = await downloadVideo(Constants.test_video_id, youtube).catch((error) => error);
assert(!(video instanceof Error), `should download video (${Constants.test_video_id})`, video);
}
const n_token = new NToken(Constants.n_scramble_sc).transform(Constants.original_ntoken);
assert(n_token == Constants.expected_ntoken, `should transform n token into ${Constants.expected_ntoken}`, n_token);
@@ -49,9 +50,12 @@ function downloadVideo(id, youtube) {
function assert(outcome, description, data) {
const pass_fail = outcome ? 'pass' : 'fail';
console.info(pass_fail, ':', description);
!outcome && (failed_tests += 1);
console.info(pass_fail, ':', description, !outcome && `\nError: ${data}` || '');
!outcome && console.error('Error: ', data);
return outcome;
};
}
performTests();