RapidAPI Scraper

What does RapidAPI Scraper do?

This RapidAPI Scraper allows you to scrape the list of APIs from RapidAPI, including each category and supports scraping without category selection. It scrapes all categories first, then scrapes the data. The data is provided in structured formats such as Excel, XML, HTML, JSON, and CSV, which you can use in your own reports, spreadsheets, and applications. RapidAPI Scraper allows you to scrape:

• Results from a RapidAPI category.
• Numbers of total builds, runs, and users.
• Detailed information about each API, including title, description, categories, and pricing model.

Why use RapidAPI Scraper?

Because RapidAPI does not provide a comprehensive API list. With RapidAPI Scraper, you can build your own products and applications on top of the RapidAPI platform.

How to use scraped RapidAPI data

• Building new applications that use the API list.
• Example: https://yourapiservice.com, etc.
• Creating a comprehensive API directory for developers.

Input parameters

If this actor is run on the Apify platform, our simple interface will help you configure all the necessary and optional parameters of this scraper before running it. This scraper recognizes the following input parameters:

• includeCategories - If set to true, the scraper will scrape the categories first and then scrape the data. If set to false, the scraper will not scrape with category (You can use Collections instead).
• categoryNames - Scrapes RapidAPI for specified categories. You can duplicate this parameter to scrape several categories simultaneously.
• maxItemsPerCategory - Sets the maximum limit for the amount of scraped results per category/collection.
• includeCollections - If set to true, the scraper will scrape the collections first and then scrape the data. If set to false, the scraper will not scrape with collection (You can use Categories instead).
• collectionNames - Scrapes RapidAPI for specified collections. You can duplicate this parameter to scrape several collections simultaneously.
• sortBy - Sorts the search query results by the specified parameter.

Example:

1{
2    "includeCategories": true,
3    "categoryNames": [
4        "Finance",
5        "Data"
6    ],
7    "includeCollections": true,
8    "collectionNames": [
9        "recommended-apis",
10        "popular-apis",
11        "list-of-free-apis"
12    ],
13    "maxItemsPerCategory": 100,
14    "sortBy": "ByTrending"
15}

Output

The output from RapidAPI Scraper is stored in a dataset. Each item contains information about an API. Here's an example of some of the output you would get if you used the input parameters above to scrape the categories "Finance":

1{
2  "source": "RapidAPI",
3  "title": "Seeking Alpha",
4  "name": "seeking-alpha",
5  "username": "apidojo",
6  "description": "Query for news, market moving, price quotes, chart, indices, analysis from investors and experts, etc...",
7  "categories": [
8    "Finance"
9  ],
10  "pictureUrl": "https://rapidapi-prod-apis.s3.amazonaws.com/be878f94-ea6d-4a87-aae2-ea43c69f155f.jpg",
11  "stats": {
12    "totalBuilds": 0,
13    "totalRuns": 0,
14    "totalUsers": 0,
15    "totalUsers7Days": 0,
16    "totalUsers30Days": 0,
17    "totalUsers90Days": 0,
18    "lastRunStartedAt": "2024-06-27T05:30:22.633Z",
19    "totalMetamorphs": 0,
20    "publicActorRunStats30Days": {
21      "ABORTED": 0,
22      "FAILED": 0,
23      "SUCCEEDED": 0,
24      "TIMED_OUT": 0,
25      "TOTAL": 0
26    }
27  },
28  "notice": "NONE",
29  "userPictureUrl": "",
30  "userFullName": "Api Dojo",
31  "currentPricingInfo": {
32    "pricingModel": "FREEMIUM"
33  },
34  "objectID": "api_ba97c871-0271-454d-9e7e-983860c54f65",
35  "updatedAt": "2024-06-27T05:30:22.633Z",
36  "score": {
37    "popularityScore": 9.9,
38    "avgLatency": 2269,
39    "avgServiceLevel": 100,
40    "avgSuccessRate": 100
41  }
42}

How much will scraping RapidAPI cost you?

When it comes to scraping, it can be challenging to estimate the resources needed to extract data as use cases may vary significantly. That's why the best course of action is to run a test scrape with a small sample of input data and limited output. You’ll get your price per scrape, which you’ll then multiply by the number of scrapes you intend to do.

Integrations and RapidAPI Scraper

Last but not least, RapidAPI Scraper can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. Or you can use webhooks to carry out an action whenever an event occurs, e.g., get a notification whenever Instagram API Scraper successfully finishes a run.

Using RapidAPI Scraper with the Apify API

The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints that enable you to manage, schedule, and run Apify actors. The API also lets you access any datasets, monitor actor performance, fetch results, create and update versions, and more.

To access the API using Node.js, use the apify-client NPM package. To access the API using Python, use the apify-client PyPI package.

Check out the Apify API reference docs for full details or click on the API tab for code examples.

Personal data

You should be aware that your results might contain personal data. Personal data is protected by GDPR in the European Union and other laws and regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers. You can read the basics of ethical web scraping in our blog post on the legality of web scraping.

Your feedback

We’re always working on improving the performance of our Actors. So if you’ve got any technical feedback for RapidAPI Scraper or simply found a bug, please create an issue on the Actor’s Issues tab in Apify Console.