Apify

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

SERP API to enable you to scrape Google Search Engine Results Pages (SERPs). Extract organic and paid results, ads, queries, People Also Ask, prices, reviews. Supports custom country or language and extraction of custom attributes. Download data as HTML table, JSON, CSV, Excel, XML.

The actor automatically sends an email to a specific address. This actor is useful for notifications and reporting. With only 3 lines of javascript code, you'll be on top of your scraping actors and never miss important results or issues.

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

This actor monitors your actors' statuses, validates their datasets' data, and displays useful information in an interactive dashboard. And if something happens, you'll get notified via email or Slack.

An example of a simple Apify actor. It takes a greeting message on input, prints it and stores a new message to output.

Create a screenshot of a website based on a specified URL. The screenshot is stored as the output in a key-value store. It can be used to monitor web changes regularly after setting up the scheduler.

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Replacement for the legacy Apify Crawler product with a backward-compatible interface. The actor uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of front-end JavaScript code.

Performs analysis of a webpage to figure the best way how to scrapes its data. On input, it takes an URL and array of strings to search for, and on output, it returns a definition of a crawler.

Example showing how to use headless Chromium with Puppeteer to open a web page, determine its dimensions, save a screenshot, and print the page to PDF. This actor must use images with Puppeteer (Node.js 8 + Puppeteer on Debian).

Run Scrapy spiders written in Python on the Apify platform.

This actor simply tests given array of URLs against selected proxy URLs or Apify proxy groups.

Example of loading a web page in headless Chrome using Selenium Webdriver.

The monitoring runner is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

The monitoring checker run status is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

The monitoring teardown is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Generates a standardized email signature for Apify team members, using provided details.

Returns a difference of two given images as JPEG or PNG image.

This example demonstrates how to use web server in actor as communication channel with outer world. Read more at Apify docs https://docs.apify.com/actors/running#container-web-server

The monitoring checker stats is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Example act using PHP as the main language.

The monitoring reporter dashboard is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

The Monitoring reporter mail is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Contains a basic boilerplate of an Apify actor with Node.js source code and a custom Dockerfile. The purpose of this actor is to help you get started quickly to create your own actors.

The monitoring checker schema is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Generates an HTTP Archive (HAR) file for web pages specified by a list of URLs in JSON or CSV file. Optionally, the pages can be loaded using proxies from a specific country. The resulting HAR files are stored in the key-value store or in the dataset.

This example demonstrates how to use web server in actor as communication channel with outer world. Read more at Apify docs https://docs.apify.com/actors/running#container-web-server

Example actor that opens a webpage with Golden Gate webcam. It takes a screenshot from the cam and saves it as output to the key-value store. You can easily use it as API that returns a screenshot with https://api.apify.com/v2/acts/apify~example-golden-gate-webcam/run-sync?token=[YOUR_API_TOKEN]

The Monitoring reporter slack is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

Iterates through all results from a crawler run and count them. Needs to be called from the crawler's finish webhook by adding an URL to finish the webhook of your crawler. Use this actor as a starting point to develop custom post-processing of data from the crawler.

The Monitoring checker duplication is a part of the Apify Monitoring Suite (apify/monitoring). See its readme for more information and how to use this.

A simple example showing how to call another actor from an actor. Note that this actor doesn't accept any input and doesn't generate any output.

The Indexer crawls recursively a website using the Puppeteer browser (headless Chrome) and indexes the selected pages to the Algolia index.

This actor simply counts up from one. In each run it prints one number. Its state (counter position) is stored in a named key-value store. The name of the store is example-counter and you can find in Apify app under Storages.

Example of an Apify actor with source code hosted in a GitHub Gist. For example, this is useful if you want to have public code examples accompanied by a working API that anyone can try straight away.