Web Scraper avatar

Web Scraper

Try for free

No credit card required

View all Actors
Web Scraper

Web Scraper

apify/web-scraper
Try for free

No credit card required

Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Do you want to learn more about this Actor?

Get a demo
CS

not crawling the TripAdvisor

Closed

capable_santoor opened this issue
6 months ago

getting the 0 reviews while scrapping the data , can you help to fix this issues?? TripAdvisor url : https://www.tripadvisor.com.au/Restaurant_Review-g1006515-d19840118-Reviews-The_Meat_Wine_Co_Chadstone-Chadstone_Monash_Greater_Melbourne_Victoria.html can you view the attachment image there most the crawling getting the 0 results data..

jindrich.bar avatar

Hello and thank you for your interest in this Actor!

This Actor is heavily dependent on your Page Function - so the issue is most likely in your setup. I tried looking into one of your runs with 0 results - the logs contain lines about "403: request blocked". The website likely knows that you're trying to scrape it with an automatic tool - see our documentation article on anti-scraping protections. Unfortunately, there is no easy way out of this - you simply have to try different configurations / tricks and see what works best for this specific website.

If you don't want to deal with all of that, you can use one of the premade Actors for scraping TripAdvisor reviews: https://apify.com/maxcopell/tripadvisor-reviews . This one is maintained by Apify and is tested and updated often, so if you just want to collect TripAdvisor review data, it might be your best bet.

Developer
Maintained by Apify

Actor Metrics

  • 2.6k monthly users

  • 222 stars

  • >99% runs succeeded

  • 45 days response time

  • Created in Mar 2019

  • Modified 3 months ago