The size to chunk the sitemap URLs into for scraping.
The timeout in milliseconds for the fetch request. Defaults to 10s.
Optional
headersThe headers to use in the fetch request.
Optional
selectorThe selector to use to extract the text from the document. Defaults to "body".
Optional
textThe text decoder to use to decode the response. Defaults to UTF-8.
Static
importsA static method that dynamically imports the Cheerio library and returns the load function. If the import fails, it throws an error.
A Promise that resolves to an object containing the load function from the Cheerio library.
Static
scrapeFetches web documents from the given array of URLs and loads them using Cheerio. It returns an array of CheerioAPI instances.
An array of URLs to fetch and load.
Optional
textDecoder: TextDecoderOptional
options: CheerioOptions & { A Promise that resolves to an array of CheerioAPI instances.
Interface representing the parameters for initializing a SitemapLoader. SitemapLoaderParams