7 days trial then $10.00/month - No credit card required now
Trustpilot reviews scraper
7 days trial then $10.00/month - No credit card required now
Easily filter and extract thousands of reviews with data such as title, description, score, reviewer, country, company response, and much more from companies on Trustpilot.com and download them to multiple file formats.
How to fully scrape all reviews rather than just a small amount
Open
Hello! Firstly, great actor :) We're pointing it at an end point with apparently 170k reviews but it is only scraping 8k approx before time out. Should we be resurrecting each time, turning off time out or some other configuration to scrape more if not all of the reviews available. Thank you for any pointers! Cheers, Dan
Hi Dan,
Thanks for reaching out and good to hear you like the actor :)
You will probably need to increase the timeout to multiple hours to scrape this much. So I suggest for the first run to disable the timeout and monitor it. The best way of resolving this would be to manually run the actor within the apify.com platform and monitor the progress there occasionally to determine how much time is needed to scrape the data. It can be difficult to determine the actual time needed to scrape this much data because the site will likely throttle the scraper.
Let me know how it goes :)
Ground-Venture-owner
Hey Casper! A belated big thanks for your quick support! We will try that
Cheers! Dan
Hi Dan,You are welcome. I hope it works for you.Sendt fra min Galaxy -------- Oprindelig besked --------Fra: Dan Keegan topic+dt6484luut5ntbcjm8@reply.apify.com Dato: 29.03.2023 15.23 (GMT+01:00) Til: Casper Rubæk deltacaspi@gmail.com Emne: Re: How to fully scrape all reviews rather than just a small amount (casper11515/trustpilot-reviews-scraper)
Hey Casper! A belated big thanks for your quick support! We will try that
Cheers!Dan
—You are receiving this email because you are subscribed to the topic.Reply to this email directly, or view the topic on Apify.
Hi Dan,
Have you had any luck with this approach?
capper1516
Hi Dan,
I just wanted to follow up and hear if you have been able to get your scraping to work? 🙂 Let me know if you need assistance.
Casper Rubæk- 49 monthly users
- 100.0% runs succeeded
- 3.8 days response time
- Created in Aug 2022
- Modified 4 days ago
