From ad1d3dbf91755978d80e22328f6a6f4bcfb33fb0 Mon Sep 17 00:00:00 2001 From: LuanRT Date: Tue, 14 Mar 2023 05:57:37 -0300 Subject: [PATCH] chore(docs): overhaul parser documentation [skip ci] --- src/parser/README.md | 232 +++++++++++++++++++++---------------------- 1 file changed, 113 insertions(+), 119 deletions(-) diff --git a/src/parser/README.md b/src/parser/README.md index 35b82682..2f49c9bc 100644 --- a/src/parser/README.md +++ b/src/parser/README.md @@ -1,38 +1,43 @@ # Parser -Sanitizes and standardizes InnerTube responses while maintaining the integrity of the data. - -Structure: - -* [`/classes`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/classes) - InnerTube nodes. -* [`/types`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/types) - General response types. -* [`/youtube`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/youtube) - Contains the logic for parsing YouTube responses. -* [`/ytmusic`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/ytmusic) - Contains the logic for parsing YouTube Music responses. -* [`/ytkids`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/ytkids) - Contains the logic for parsing YouTube Kids responses. -* [`helpers.ts`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/helpers.ts) - Helper functions/classes for the parser. -* [`parser.ts`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/parser.ts) - The core of the parser. -* [`map.ts`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/map.ts) - A list of all InnerTube nodes, it is used to determine which node to use for a given renderer. Note that this file is auto-generated and should not be edited manually. +The parser is responsible for sanitizing and standardizing InnerTube responses while preserving the integrity of the data. ## Table of Contents -
    -
  1. - API -
    2. -
  2. - Usage - -
    3. -
  3. Adding new nodes
    4. -
  4. How it works
    5. -
+- [Parser](#parser) + - [Table of Contents](#table-of-contents) + - [Structure](#structure) + - [Core](#core) + - [Clients](#clients) + - [API](#api) + - [`parse(data?: RawData, requireArray?: boolean, validTypes?: YTNodeConstructor | YTNodeConstructor[])`](#parsedata-rawdata-requirearray-boolean-validtypes-ytnodeconstructort--ytnodeconstructort) + - [`parseResponse(data: IRawResponse): T`](#parseresponsedata-irawresponse-t) + - [Usage](#usage) + - [ObservedArray](#observedarray) + - [SuperParsedResponse](#superparsedresponse) + - [YTNode](#ytnode) + - [Type Casting](#type-casting) + - [Accessing properties without casting](#accessing-properties-without-casting) + - [Memo](#memo) + - [Adding new nodes](#adding-new-nodes) + - [How it works](#how-it-works) -___ +## Structure +### Core + +* [`/types`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/types) - General response types. +* [`/classes`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/classes) - InnerTube nodes. +* [`helpers.ts`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/helpers.ts) - Helper functions/classes for the parser. +* [`parser.ts`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/parser.ts) - The core of the parser. +* [`map.ts`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/map.ts) - Contains a list of all the InnerTube nodes, which is used to determine the appropriate node for a given renderer. It's important to note that this file is automatically generated and should not be edited manually. + +### Clients + +The parser itself is not tied to any specific client. Therefore, we have a separate folder for each client that the library supports. These folders are responsible for arranging the parsed data into a format that can be easily consumed and understood. Additionally, the underlying data is also exposed for those who wish to access it. + +* [`/youtube`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/youtube) +* [`/ytmusic`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/ytmusic) +* [`/ytkids`](https://github.com/LuanRT/YouTube.js/blob/main/src/parser/ytkids) ## API @@ -44,86 +49,82 @@ ___ -#### parse(data, requireArray, validTypes) +### `parse(data?: RawData, requireArray?: boolean, validTypes?: YTNodeConstructor | YTNodeConstructor[])` Responsible for parsing individual nodes. | Param | Type | Description | | --- | --- | --- | -| data | `any` | The data | +| data | `RawData` | The data to parse | | requireArray | `?boolean` | Whether the response should be an array | | validTypes | `YTNodeConstructor \| YTNodeConstructor[] \| undefined` | Types of `YTNode` allowed | -When `requireArray` is `true`, the response will be an `ObservedArray`. +- If `requireArray` is `true`, the response will be an `ObservedArray`. +- If `validTypes` is `undefined`, the response will be an array of YTNodes. +- If `validTypes` is an array, the response will be an array of YTNodes that are of the types specified in the array. +- If `validTypes` is a single type, the response will be an array of YTNodes that are of the type specified. -When `validTypes` is `undefined`, the response will be an array of YTNodes. - -When `validTypes` is an array, the response will be an array of YTNodes that are of the types specified in the array. - -When `validTypes` is a single type, the response will be an array of YTNodes that are of the type specified. - -If you do not specify `requireArray`, the return type of the function will not be known at runtime, and therefore we return the response wrapped in a helper, `SuperParsedResponse`, to gain access to the response. +If you do not specify `requireArray`, the return type of the function will not be known at runtime. Therefore, to gain access to the response, we return it wrapped in a helper, `SuperParsedResponse`. You may use the `Parser#parseArray` and `Parser#parseItem` methods to parse the response in a deterministic way. -#### parseResponse(data) + +### `parseResponse(data: IRawResponse): T` Unlike `parse`, this can be used to parse the entire response object. | Param | Type | Description | | --- | --- | --- | -| data | `object` | Raw InnerTube response | +| data | `IRawResponse` | Raw InnerTube response | ## Usage -## ObservedArray -You may use `ObservedArray` as a normal array, but it provides additional methods for typesafe access and casting. +### ObservedArray +You can utilize an `ObservedArray` as a regular array, but it also offers further methods for accessing and casting values in a type-safe manner. ```ts // For example, we have a feed, and want all the videos: const feed = new ObservedArray([...feed.contents]); -const videos = feed.filterType(GridVideo); -// This is now a GridVideo[] -// Or we want only the first video: +// Here, we use the filterType method to retrieve only GridVideo items from the feed. +const videos = feed.filterType(GridVideo); +// `videos` is now a GridVideo[] array. + +// Alternatively, we can use firstOfType to retrieve the first GridVideo item from the feed. const firstVideo = feed.firstOfType(GridVideo); -// We may cast the whole array to a GridVideo[] and throw if we have any non-GridVideo elements: +// If we want to make sure that all elements in the `feed` array are of the `GridVideo` type, we can use the `as` method to cast the entire array to a `GridVideo[]` type. If the cast fails because of non-GridVideo items, an exception is thrown. const allVideos = feed.as(GridVideo); -// There are some extra methods for ObservedArray -// which we use internally but not documented here (yet). -// see the source code for more details. +// Note that ObservedArray provides additional methods beyond what's shown here, which we use internally. For more information, see the source code or documentation. ``` -## SuperParsedResponse -Represents a parsed response in an unknown state. Either a `YTNode` or an `ObservedArray` or `null`. - -You will need to assert the type and unwrap the response to get the actual value. +### SuperParsedResponse +Represents a parsed response in an unknown state. Either a `YTNode`, an `ObservedArray`, or `null`. To extract the actual value, you must first assert the type and unwrap the response. ```ts -// We can assert we have a YTNode: +// First, parse the data and store it in `response`. const response = Parser.parse(data); + +// Check whether `response` is a YTNode. if (response.is_item) { + // If so, we can assert that it is a YTNode and retrieve it. const node = response.item(); } -// We can assert we have an ObservedArray: -const response = Parser.parse(data); +// Check whether `response` is an ObservedArray. if (response.is_array) { + // If so, we can assert that it is an ObservedArray and retrieve its contents as an array of YTNode objects. const nodes = response.array(); } -// Or lastly a null response: -const response = Parser.parse(data); +// Finally, to check if `response` is a null value, use the `is_null` getter. const is_null = response.is_null; ``` -## YTNode -All renderers returned by InnerTube are converted to this generic class and then extended for the specific renderers. - -This class is what allows us a typesafe way to use data returned by the InnerTube API. +### YTNode +All renderers returned by InnerTube are converted to this generic class and then extended for the specific renderers. This class is what allows us a type-safe way to use data returned by the InnerTube API. Here's how to use this class to access returned data: @@ -131,10 +132,10 @@ Here's how to use this class to access returned data: ```ts // We can cast a YTNode to a child class of YTNode const results = node.as(TwoColumnSearchResults); -// This will throw if the node is not a TwoColumnSearchResults -// We thus may want to check for the type of the node before casting +// This will throw an error if the node is not a TwoColumnSearchResults. +// Therefore, we may want to check for the type of the node before casting. if (node.is(TwoColumnSearchResults)) { - // We do not need to recast the node, it is already a TwoColumnSearchResults after calling is() and using it in the branch where is() returns true + // We do not need to recast the node; it is already a TwoColumnSearchResults after calling is() and using it in the branch where is() returns true. const results = node; } @@ -144,21 +145,20 @@ const results = node.as(TwoColumnSearchResults, VideoList); // Similarly, we can check if the node is of a certain type. if (node.is(TwoColumnSearchResults, VideoList)) { - // Again no casting is needed, the node is already of the correct type. + // // Again, no casting is needed; the node is already of the correct type. const results = node; } ``` ### Accessing properties without casting -Sometimes multiple nodes have the same properties and we don't want to check the type of the node before accessing the property, for example, the property "contents" is used by many node types, and we may add more in the future, as such we want to only assert the property instead of casting to a specific type. +Sometimes multiple nodes have the same properties, and we don't want to check the type of the node before accessing the property. For example, the property 'contents' is used by many node types, and we may add more in the future. As such, we want to only assert the property instead of casting to a specific type. ```ts -// Accessing a property on a node which you aren't sure if it exists. +// Accessing a property on a node when you aren't sure if it exists. const prop = node.key("contents"); -// This returns the value wrapped into a `Maybe` type -// which you can use to find the type of the value -// note, however, this throws an error if the key doesn't exist -// we may want to check for the key before accessing it. + +// This returns the value wrapped into a `Maybe` type, which you can use to determine the type of the value. +// However, this throws an error if the key doesn't exist, so we may want to check for the key before accessing it. if (node.hasKey("contents")) { const prop = node.key("contents"); } @@ -169,19 +169,18 @@ if (prop.isString()) { const value = prop.string(); } -// We can do more complex assertions too, -// like checking for instanceof. +// We can do more complex assertions, like checking for instanceof. const prop = node.key("contents"); -if (prop.isInstanceof(Text)) { - const text = prop.instanceof(Text); - // and then use the value as the given type +if (prop.isInstanceOf(Text)) { + const text = prop.instanceOf(Text); + + // Then use the value as the given type. text.runs.forEach(run => { console.log(run.text); }); } -// There are some special methods for using with the parser — -// such as getting the value as a YTNode. +// There are special methods for use with the parser, such as getting the value as a YTNode. const prop = node.key("contents"); if (prop.isNode()) { const node = prop.node(); @@ -200,13 +199,12 @@ if (prop.isNodeOfType([TwoColumnSearchResults, VideoList])) { } // Sometimes an ObservedArray is returned when working with parsed data. -// We've got a helper for that too; +// We also have a helper for this. const prop = node.key("contents"); if (prop.isObserved()) { const array = prop.observed(); - // Now we may use all the ObservedArray methods as normal, - // like finding nodes of a certain type for example. + // Now we can use all the ObservedArray methods as normal, such as finding nodes of a certain type. const results = array.filterType(GridVideo); } @@ -215,8 +213,8 @@ const prop = node.key("contents"); if (prop.isParsed()) { const result = prop.parsed(); - // SuperParsedResult is another helper for typesafe access to the parsed data, - // it is explained above with the `Parser#parse` method. + // SuperParsedResult is another helper for type-safe access to the parsed data. + // It is explained above with the `Parser#parse` method. const results = results.array(); const videos = results.filterType(Video); } @@ -226,51 +224,47 @@ if (prop.isParsed()) { const prop = node.key("contents"); const value = prop.any(); -// Arrays are also a special case as every element may be of a different type, -// the `arrayOfMaybe` method will return an array of `Maybe`s. +// Arrays are a special case, as every element may be of a different type. +// The `arrayOfMaybe` method will return an array of `Maybe`s. const prop = node.key("contents"); if (prop.isArray()) { const array = prop.arrayOfMaybe(); - // This will return Maybe[] + // This will return `Maybe[]`. } -// Or if you want zero type safety you can use the `array` method. +// Or, if you don't need type safety, you can use the `array` method. const prop = node.key("contents"); if (prop.isArray()) { const array = prop.array(); - // This will return any[] + // This will return any[]. } ``` -## Memo -The `Memo` class is a helper class for memoizing values in the `Parser#parseResponse` method. It is useful for finding nodes after parsing the response. +### Memo +The `Memo` class is a helper class for memoizing values in the `Parser#parseResponse` method. It can be used to conveniently access nodes after parsing the response. -Say we want all of the videos in a search result. We can use the `Memo` to find all of them quickly without recursing through the response. +For example, if we'd like to obtain all of the videos from a search result, we can use the `Memo#getType` method to find them quickly without needing to traverse the entire response. ```ts const response = Parser.parseResponse(data); const videos = response.contents_memo.getType(Video); -// This returns the nodes as a ObservedArray