Zillow Explorer
7 days trial then $30.00/month - No credit card required now
Zillow Explorer
7 days trial then $30.00/month - No credit card required now
💫 Scrape Zillow.com Homes
Can we make concurrent runs?
Closed
Hi I have made some runs where I have tried to call the scraper in parallel with different zipcode inputs. Right now I am just doing 3 at a time but was hoping to move up to my concurrency limit of 32. It appears that even though they all start at the same time, they still run and finish sequentially.
Is concurrent runs not something that your scraper supports?
data_scraper123
I am closing this because I can't reproduce consistently. However, this happened when trying to run concurrent runs each with a list of 1000 zpids as input. Many of the responses fail in this case, and you get on average about 400 successful responses. When I reduced it to 100 zpids as input, no responses failed and the runs properly ran in parallel. Interesting behavior but still works effectively for my use case. My guess is that having to recycle proxies or do retries is affecting the concurrent behavior
Hi @data_scraper123 thank you for reporting. The concurrent limit for this actor is max=3 per user. we don't want to hit zillow with too many requests, at once. about 1000 zpids, let me look into it.
catActor Metrics
16 monthly users
-
7 stars
98% runs succeeded
22 days response time
Created in Jul 2022
Modified 3 months ago
