You can access the Scrapy Cloud Runner programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "spiderName": "page_meta",
12    "startUrls": [
13        {
14            "url": "https://apify.com"
15        }
16    ],
17    "spiderArgs": [
18        {
19            "key": "category",
20            "value": "books"
21        }
22    ]
23};
24
25// Run the Actor and wait for it to finish
26const run = await client.actor("solutionssmart/scrapy-cloud-runner").call(input);
27
28// Fetch and print Actor results from the run's dataset (if any)
29console.log('Results from dataset');
30console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
31const { items } = await client.dataset(run.defaultDatasetId).listItems();
32items.forEach((item) => {
33    console.dir(item);
34});
35
36// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Scrapy Cloud Runner – Run Python Scrapy Spiders on Apify API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Scrapy Cloud Runner API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Scrapy Cloud Runner API in Python

Scrapy Cloud Runner API through CLI

Scrapy Cloud Runner OpenAPI definition

Scrapy Cloud Runner API

Scrapy Cloud Runner

sovanza.inc/scrapy-cloud-runner

Scrapy Cloud Runner runs Scrapy spiders on Apify with runtime arguments, custom settings, schedules, webhooks, and automatic dataset export. It supports custom spiders, compact JSON output, and JSON/CSV/Excel dataset downloads.

Sovanza

Python Scrapy template

ellustar/python-scrapy-template

“A ready-to-use Python Scrapy template designed for building fast and scalable data extraction actors. Includes a clean project structure, example spiders, settings configuration, and best practices to help developers quickly create, customize, and deploy Scrapy-based workflows.”

Ellustar

Python Scrapy template

ellustar/my-actor-41

A Python Scrapy template designed to extract actor names efficiently from websites. It includes structured spiders, customizable selectors, and clean output handling, making it easy to scrape, store, and scale actor data for research, media, or automation projects.

Ellustar

Cloud Details Spider

getdataforme/cloud-details-spider

Cloud Details Spider extracts comprehensive details from cloud service pages, including titles, features, pricing, and links, delivering structured JSON for easy analysis. Supports batch URL processing, reliable parsing, and seamless Apify integration, ideal for research, monitoring, and automation.

GetDataForMe

Scrapy Books Example

vdusek/scrapy-books-example

Example of Python Scrapy project. It scrapes book data from https://books.toscrape.com/.

Vlada Dusek

Selenium Cloud Runner

sovanza.inc/selenium-cloud-runner

Selenium Cloud Runner scrapes JavaScript-heavy websites using Selenium and headless Chrome. It extracts data with CSS or XPath rules, supports scrolling, popup handling, screenshots, proxies, retries, and structured dataset exports.

Sovanza

5.0