Pricing

Pay per usage

Vanilla JS Scraper

Scrape the web using familiar JavaScript methods! Crawls websites using raw HTTP requests, parses the HTML with the JSDOM package, and extracts data from the pages using Node.js code. Supports both recursive crawling and lists of URLs. This actor is a non jQuery alternative to CheerioScraper.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Matthias Stephens

Actor stats

Bookmarked

515

Total users

Monthly active users

2 years ago

Last modified

Categories

Automation

Developer tools

Open source

Requests

requests

Required

A static list of URLs to scrape.

For details, see the Start URLs section in the README.

Type:array

Pseudo-URLs

pseudoUrls

Optional

Specifies what kind of URLs found by the Link selector should be added to the request queue. A pseudo-URL is a URL with regular expressions enclosed in [] brackets, e.g. http://www.example.com/[.*].

If Pseudo-URLs are omitted, the actor enqueues all links matched by the Link selector.

For details, see Pseudo-URLs in README.

Type:array

Default:

[]

Link selector

linkSelector

Optional

A CSS selector stating which links on the page (<a> elements with href attribute) shall be followed and added to the request queue. To filter the links added to the queue, use the Pseudo-URLs field.

If the Link selector is empty, the page links are ignored.

For details, see the Link selector in README.

Type:string

Page function

pageFunction

Required

A JavaScript function that is executed for every page loaded server-side in Node.js 12. Use it to scrape data from the page, perform actions or add new URLs to the request queue.

For details, see Page function in README.

Type:string

Pre-navigation hooks

preNavigationHooks

Optional

Async functions that are sequentially evaluated before the navigation. Good for setting additional cookies or browser properties before navigation. The function accepts two parameters, crawlingContext and requestAsBrowserOptions, which are passed to the requestAsBrowser() function the crawler calls to navigate.

Type:string

Post-navigation hooks

postNavigationHooks

Optional

Async functions that are sequentially evaluated after the navigation. Good for checking if the navigation was successful. The function accepts crawlingContext as the only parameter.

Type:string

Proxy configuration

proxy

Optional

Specifies proxy servers that will be used by the scraper in order to hide its origin.

For details, see Proxy configuration in README.

Type:object

Default:

{
  "useApifyProxy": false
}

Debug log

debug

Optional

Include debug messages in the log?

Type:boolean

Default:false

Max concurrency

maxConcurrency

Optional

Specifies the maximum number of pages that can be processed by the scraper in parallel. The scraper automatically increases and decreases concurrency based on available system resources. This option enables you to set an upper limit, for example to reduce the load on a target web server.

Type:integer

Minimum:1

Default:50

Max request retries

maxRequestRetries

Optional

The maximum number of times the scraper will retry to load each web page on error, in case of a page load error or an exception thrown by the Page function.

If set to 0, the page will be considered failed right after the first error.

Type:integer

Minimum:0

Default:3

Page load timeout

pageLoadTimeoutSecs

Optional

The maximum amount of time the scraper will wait for a web page to load, in seconds. If the web page does not load in this timeframe, it is considered to have failed and will be retried (subject to Max page retries), similarly as with other page load errors.

Type:integer

Minimum:1

Default:60

Page function timeout

pageFunctionTimeoutSecs

Optional

The maximum amount of time the scraper will wait for the Page function to execute, in seconds. It is always a good idea to set this limit, to ensure that unexpected behavior in page function will not get the scraper stuck.

Type:integer

Minimum:1

Default:60

Ignore SSL errors

ignoreSslErrors

Optional

If enabled, the scraper will ignore SSL/TLS certificate errors. Use at your own risk.

Type:boolean

Default:false

Additional MIME types

additionalMimeTypes

Optional

A JSON array specifying additional MIME content types of web pages to support. By default, Cheerio Scraper supports the text/html and application/xhtml+xml content types, and skips all other resources. For details, see Content types in README.

Type:array

Default:

[]

Dataset name

datasetName

Optional

Name or ID of the dataset that will be used for storing results. If left empty, the default dataset of the run will be used.

Type:string

Key-value store name

keyValueStoreName

Optional

Name or ID of the key-value store that will be used for storing records. If left empty, the default key-value store of the run will be used.

Type:string

Custom data

customData

Optional

A custom JSON object that is passed to the Page function as context.customData. This setting is useful when invoking the scraper via API, in order to pass some arbitrary parameters to your code.

Type:object

Default:

{}

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

14K

5.0

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

974

4.2

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

13K

5.0

Web Scraper

apify/web-scraper

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

109K

4.8

bcv-tasa-oficial

grupoaceivzla/bcv-tasa-oficial

Grupo ACEI

Playwright Scraper

apify/playwright-scraper

Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

5.9K

2.8

Camoufox Scraper

apify/camoufox-scraper

Crawls websites with stealthy Camoufox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

189

5.0

Nodejs Runner

martin.forejt/nodejs-runner

This Actor allows you to quickly run arbitrary JavaScript code in a real Node.js environment, making it ideal for testing, debugging, or executing small scripts without setting up a local Node.js instance. The actor spawns a separate Node.js process to run the provided code and captures the logs.

Martin Forejt

162

JSDOM Scraper

apify/jsdom-scraper

Parses the HTML using the JSDOM library, providing the same DOM API as browsers do (e.g. `window`). It is able to process client-side JavaScript without using a real browser. Performance-wise, it stands somewhere between the Cheerio Scraper and the browser scrapers.

Apify

140

4.3

Npm Package Scraper

openclawmara/npm-package-scraper

Scrape npm the JavaScript package registry. Search packages, extract metadata, download statistics, dependencies, and version history. Track package popularity trends. Essential for JavaScript and Node.js ecosystem research and dependency analysis.