Scrapy Executor
DeprecatedView all Actors
This Actor is deprecated
This Actor is unavailable because the developer has decided to deprecate it. Would you like to try a similar Actor instead?
See alternative ActorsScrapy Executor
apify/scrapy-executor
Run Scrapy spiders written in Python on the Apify platform.
The code examples below show how to run the Actor and get its results. To run the code, you need to have an Apify account. Replace <YOUR_API_TOKEN> in the code with your API token, which you can find under Settings > Integrations in Apify Console. Learn mode
Node.js
Python
curl
1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4const client = new ApifyClient({
5 token: '<YOUR_API_TOKEN>',
6});
7
8// Prepare Actor input
9const input = {
10 "scrapyCode": "import scrapy\nimport apify\n \nclass MySpider(scrapy.Spider):\n \tname = 'apifySpider'\n \n \tdef start_requests(self):\n \t\turls = [\n \t\t\t'https://apify.com',\n \t\t\t'https://apify.com/store',\n \t\t]\n \t\tfor url in urls:\n \t\t\tyield scrapy.Request(url=url, callback=self.parse)\n \n \tdef parse(self, response):\n \t\turl = response.url\n \t\ttitle = response.css('title::text').get()\n \t\toutput = {\n \t\t\t'url': url,\n \t\t\t'title': title\n \t\t}\n \t\tapify.pushData(output)"
11};
12
13(async () => {
14 // Run the Actor and wait for it to finish
15 const run = await client.actor("apify/scrapy-executor").call(input);
16
17 // Fetch and print Actor results from the run's dataset (if any)
18 console.log('Results from dataset');
19 const { items } = await client.dataset(run.defaultDatasetId).listItems();
20 items.forEach((item) => {
21 console.dir(item);
22 });
23})();Developer
ApifyMaintained by Community
Categories
