3 days trial then $10.00/month - No credit card required now
Advanced Google Maps Reviews Scraper
3 days trial then $10.00/month - No credit card required now
This Google Maps Reviews scraper lets you collect reviews for places from public Google Maps pages. You can get the star rating, review text, photos and images, response from the place, and (optionally) some details about the reviewer like their name and a link to their profile.
Erroring Out before getting all reviews
Closed
2024-02-26T04:04:43.483Z WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. requestHandler timed out after 60 seconds. {"id":"1ZxgH7mx6RXYFg2","url":"https://www.google.com/maps/place/?q=place_id:XXXXXXXXXXXXXXXXXX","retryCount":1} 2024-02-26T04:04:43.497Z WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. requestHandler timed out after 60 seconds. {"id":"FsWrhvZNgvxHQiw","url":"https://www.google.com/maps/place/?q=place_id:XXXXXXXXXXXXXXXXX","retryCount":1} 2024-02-26T04:04:43.585Z INFO PlaywrightCrawler: Error getting reviews for XXXXXXXXXXX. Error: Error: locator.count: Target page, context or browser has been closed
For public concerns, I replaced with XXXXXXXXXXXX Always errors out before getting all reviews for a place with 540 reviews. Using max of 4096 memory. Using US Residential Proxy. Any ideas?
overjoyed_ravel
Update - seems there is a 60-second built-in timeout error being thrown (ignoring my setting of 3600 for the overall process). It only ever gets a partial set of reviews, then repeats itself, instead of starting from the last processed review. So, we need to increase whatever is causing the 60 second timeout, or the config needs to be updated to resume from the last review it successfully processed.
2024-02-28T13:29:56.972Z INFO PlaywrightCrawler: All requests from the queue have been processed, the crawler will shut down. 2024-02-28T13:29:57.390Z INFO PlaywrightCrawler: Final request statistics: {"requestsFinished":0,"requestsFailed":1,"retryHistogram":[null,null,null,1],"requestAvgFailedDurationMillis":62932,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":62932,"requestsTotal":1,"crawlerRuntimeMillis":261411} 2024-02-28T13:29:57.391Z INFO PlaywrightCrawler: Error analysis: {"totalErrors":1,"uniqueErrors":1,"mostCommonErrors":["1x: requestHandler timed out after 60 seconds. (/home/myuser/node_modules/@apify/timeout/index.js:62:68)"]} 2024-02-28T13:29:57.392Z INFO PlaywrightCrawler: Finished! Total 1 requests: 0 succeeded, 1 failed. {"terminal":true}
And it should not be reporting success in this case. However your actor is reporting success, so it is very hard to know when errors are occurring.
overjoyed_ravel
hello developer? We are paying for an actor, 5 days without support is absurd
Apologies for the confusion here, we can't currently see your paid subscription, hence your ticket was not prioritised. Could you please confirm when you signed up so we can track it down? We're working on the issue in parallel.
overjoyed_ravel
I was on a trial. If you guys can fix it, and reset my trial, so I can verify before subscribing?
overjoyed_ravel
I went ahead and paid for it... please fix the 60 second timeout internal to the process.
overjoyed_ravel
I just realized we dont have review date. Is that something not possible to get?
Review date isn't present on the page so it's not possible to scrape. You could approximate it using time elapsed - that will be more accurate for recent reviews, but the longer ago the review was left, the less accurate it'll be.
This has been corrected and the default 60 second timeout that Apify applies has been overridden. I've just checked it with a replication of the run that failed for you; it's no longer timing out, and all reviews are fetched. Please be cautious with setting a very high number of max reviews, since you could have a very long-running process.
overjoyed_ravel
Ok, this is fixed. Thank you.
slash scrape- 15 monthly users
- 2 stars
- 100.0% runs succeeded
- 4.7 days response time
- Created in Dec 2023
- Modified 4 days ago
