What does Apify Scraper do?

This Apify Scraper allows you to scrape the list of Actors/APIs from Apify, including each category and supports scraping without category selection. It scrapes all categories first, then scrapes the data. The data is provided in structured formats such as Excel, XML, HTML, JSON, and CSV, which you can use in your own reports, spreadsheets, and applications. Apify Scraper allows you to scrape:

• Results from a Apify category.
• Numbers of total builds, runs, stars and users.
• Detailed information about each API, including title, description, categories, and pricing model.

Why use Apify Scraper?

Because Apify does not provide a comprehensive API list. With Apify Scraper, you can build your own products and applications on top of the Apify platform.

How to use scraped Apify data

• Building new applications that use the API list.
• Example: https://yourapiservice.com, etc.
• Creating a comprehensive API directory for developers.

Input parameters

If this actor is run on the Apify platform, our simple interface will help you configure all the necessary and optional parameters of this scraper before running it. This scraper recognizes the following input parameters:

• categoryNames - Scrapes Apify for specified categories. You can duplicate this parameter to scrape several categories simultaneously.
• maxItemsPerCategory - Sets the maximum limit for the amount of scraped results per category/collection.
• sortBy - Sorts the search query results by the specified parameter.
• pricingModel - Filters the search query results by the specified pricing model.
• managedBy - Filters the search query results by the specified maintainer (Apify or Community).

Example:

1{
2    "categoryNames": [
3        "marketing",
4        "ai",
5        "jobs"
6    ],
7    "maxItemsPerCategory": 1000,
8    "sortBy": "ByRelevance",
9    "pricingModel": "all",
10    "managedBy": "all"
11}

Output

The output from Apify Scraper is stored in a dataset. Each item contains information about an API. Here's an example of some of the output you would get if you used the input parameters above to scrape the categories "Finance":

1{
2    "source": "Apify",
3    "title": "🔥 LinkedIn Jobs Scraper",
4    "name": "linkedin-jobs-scraper",
5    "username": "bebity",
6    "description": "ℹ️ Designed for both personal and professional use, simply enter your desired job title and location to receive a tailored list of job opportunities. Try it today!",
7    "categories": [
8        "Jobs",
9        "Automation",
10        "Business"
11    ],
12    "pictureUrl": "https://apify-image-uploads-prod.s3.amazonaws.com/BHzefUZlZRKWxkTck/NophtTwbi2mrFvSqa-LinkedIn_icon_circle.svg.png",
13    "objectId": "",
14    "stats": {
15        "totalBuilds": 112,
16        "totalRuns": 645874,
17        "totalUsers": 2625,
18        "totalUsers7Days": 124,
19        "totalUsers30Days": 347,
20        "totalUsers90Days": 847,
21        "lastRunStartedAt": "2024-08-19T23:33:01.285Z",
22        "publicActorRunStats30Days": {
23            "ABORTED": 170,
24            "FAILED": 0,
25            "SUCCEEDED": 31578,
26            "TIMED-OUT": 49,
27            "TOTAL": 31797
28        }
29    },
30    "notice": "NONE",
31    "totalStars": 29,
32    "userPictureUrl": "https://images.apifyusercontent.com/NIO86SxxBSCgbloI3Bs-vajk4mNErvJarMbmzLw5e6g/rs:fill:32:32/aHR0cHM6Ly9hcGlmeS1pbWFnZS11cGxvYWRzLXByb2QuczMuYW1hem9uYXdzLmNvbS9OdFN5dXVlTEE1OVc2dWtMTS9wSE5nN1ozUVNqOXVHR0xZVC1iZWJpdHlfbG9nb18lMjgxJTI5LnBuZw.png",
33    "userFullName": "Bebity",
34    "currentPricingInfo": {
35        "pricingModel": "FLAT_PRICE_PER_MONTH",
36        "startedAt": "2023-10-25T14:02:55.081Z",
37        "pricePerUnitUsd": 29.99,
38        "trialMinutes": 4320,
39        "createdAt": "2023-10-11T14:06:09.050Z",
40        "apifyMarginPercentage": 0,
41        "notifiedAboutFutureChangeAt": "2023-10-11T14:06:35.092Z",
42        "notifiedAboutChangeAt": "2023-10-25T15:06:21.725Z"
43    },
44    "objectID": "BHzefUZlZRKWxkTck"
45}

How much will scraping Apify cost you?

Apify Scraper is billed based on the results of the scrape. 1$ for each 1000 APIs/Actors (0.001$ per result). The price per scrape is calculated based on the number of scraped successfully.

Integrations and Apify Scraper

Last but not least, Apify Scraper can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. Or you can use webhooks to carry out an action whenever an event occurs, e.g., get a notification whenever Instagram API Scraper successfully finishes a run.

Using Apify Scraper with the Apify API

The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints that enable you to manage, schedule, and run Apify actors. The API also lets you access any datasets, monitor actor performance, fetch results, create and update versions, and more.

To access the API using Node.js, use the apify-client NPM package. To access the API using Python, use the apify-client PyPI package.

Check out the Apify API reference docs for full details or click on the API tab for code examples.

Personal data

You should be aware that your results might contain personal data. Personal data is protected by GDPR in the European Union and other laws and regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. You can read the basics of ethical web scraping in our blog post on the legality of web scraping.

Your feedback

We’re always working on improving the performance of our Actors. So if you’ve got any technical feedback for Apify Scraper or simply found a bug, please create an issue on the Actor’s Issues tab in Apify Console.