Trustpilot reviews scraper avatar
Trustpilot reviews scraper
Try for free

7 days trial then $10.00/month - No credit card required now

View all Actors
Trustpilot reviews scraper

Trustpilot reviews scraper

casper11515/trustpilot-reviews-scraper
Try for free

7 days trial then $10.00/month - No credit card required now

Easily filter and extract thousands of reviews with data such as title, description, score, reviewer, country, company response, and much more from companies on Trustpilot.com and download them to multiple file formats.

User avatar

How to fully scrape all reviews rather than just a small amount

Open

Ground-Venture opened this issue
a year ago

Hello! Firstly, great actor :) We're pointing it at an end point with apparently 170k reviews but it is only scraping 8k approx before time out. Should we be resurrecting each time, turning off time out or some other configuration to scrape more if not all of the reviews available. Thank you for any pointers! Cheers, Dan

User avatar

Hi Dan,

Thanks for reaching out and good to hear you like the actor :)

You will probably need to increase the timeout to multiple hours to scrape this much. So I suggest for the first run to disable the timeout and monitor it. The best way of resolving this would be to manually run the actor within the apify.com platform and monitor the progress there occasionally to determine how much time is needed to scrape the data. It can be difficult to determine the actual time needed to scrape this much data because the site will likely throttle the scraper.

Let me know how it goes :)

User avatar

Ground-Venture-owner

a year ago

Hey Casper! A belated big thanks for your quick support! We will try that

Cheers! Dan

User avatar

Hi Dan,You are welcome. I hope it works for you.Sendt fra min Galaxy -------- Oprindelig besked --------Fra: Dan Keegan topic+dt6484luut5ntbcjm8@reply.apify.com Dato: 29.03.2023 15.23 (GMT+01:00) Til: Casper Rubæk deltacaspi@gmail.com Emne: Re: How to fully scrape all reviews rather than just a small amount (casper11515/trustpilot-reviews-scraper)

Hey Casper! A belated big thanks for your quick support! We will try that

Cheers!Dan

—You are receiving this email because you are subscribed to the topic.Reply to this email directly, or view the topic on Apify.
User avatar

Hi Dan,

Have you had any luck with this approach?

User avatar

capper1516

a year ago

Hi Dan,

I just wanted to follow up and hear if you have been able to get your scraping to work? 🙂 Let me know if you need assistance.

Developer
Maintained by Community
Actor metrics
  • 41 monthly users
  • 99.6% runs succeeded
  • 3.9 days response time
  • Created in Aug 2022
  • Modified about 1 month ago
Categories