Shopify Scraper avatar
Shopify Scraper
Try for free

No credit card required

View all Actors
Shopify Scraper

Shopify Scraper

pocesar/shopify-scraper
Try for free

No credit card required

Automate monitoring prices on the most popular solution for building online stores and selling products online. Crawl arbitrary Shopify-powered online stores and extract a list of all products in a structured form, including product title, price, description, etc.

User avatar

startURLs - Only First

Open

gold_hat opened this issue
2 years ago

When running this actor with multiple inputs for the startURL only the first URL is used to make request and return data, the others are not mentioned or requested? I have provided my logs, I am only using three URLS

2022-08-15T19:13:38.346Z ACTOR: Pulling Docker image from repository. 2022-08-15T19:13:38.453Z ACTOR: Creating Docker container. 2022-08-15T19:13:38.499Z ACTOR: Starting Docker container. 2022-08-15T19:13:42.997Z INFO System info {"apifyVersion":"2.3.2","apifyClientVersion":"2.4.1","osType":"Linux","nodeVersion":"v16.15.0"} 2022-08-15T19:13:45.388Z INFO BasicCrawler:AutoscaledPool: state {"currentConcurrency":0,"desiredConcurrency":2,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":null},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":null},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":null},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":null}}} 2022-08-15T19:13:48.799Z INFO BasicCrawler: All the requests from request list and/or request queue have been processed, the crawler will shut down. 2022-08-15T19:13:49.039Z INFO BasicCrawler: Final request statistics: {"requestsFinished":6,"requestsFailed":0,"retryHistogram":[6],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":513,"requestsFinishedPerMinute":97,"requestsFailedPerMinute":0,"requestTotalDurationMillis":3076,"requestsTotal":6,"crawlerRuntimeMillis":3719} 2022-08-15T19:13:49.042Z INFO Found 10 URLs from 4 sitemap URLs 2022-08-15T19:13:49.521Z INFO CheerioCrawler:AutoscaledPool: state {"currentConcurrency":0,"desiredConcurrency":2,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":null},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.7,"actualRatio":null},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":null},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":null}}} 2022-08-15T19:13:55.755Z INFO CheerioCrawler: Crawler reached the maxRequestsPerCrawl limit of 10 requests and will shut down soon. Requests that are in progress will be allowed to finish. 2022-08-15T19:13:55.758Z INFO CheerioCrawler: Earlier, the crawler reached the maxRequestsPerCrawl limit of 10 requests and all requests that were in progress at that time have now finished. In total, the crawler processed 10 requests and will shut down. 2022-08-15T19:13:56.026Z INFO CheerioCrawler: Final request statistics: {"requestsFinished":10,"requestsFailed":0,"retryHistogram":[10],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":1060,"requestsFinishedPerMinute":91,"requestsFailedPerMinute":0,"requestTotalDurationMillis":10601,"requestsTotal":10,"crawlerRuntimeMillis":6569}

"startUrls": [ { "url": "https://cbdfarmacyca.com/collections/all-products" }, { "url": "https://cleangreendelivers.com/collections/all-products" }, { "url": "https://friendly-strangers.com/collections/all-products" } ]

Developer
Maintained by Community
Actor metrics
  • 58 monthly users
  • 98.4% runs succeeded
  • 28.0 days response time
  • Created in Jul 2021
  • Modified 11 months ago
Categories