1 day trial then $20.00/month - No credit card required now
TripAdvisor Scraper
1 day trial then $20.00/month - No credit card required now
Explore with our Trip Advisor Scraper: an easy search for hotels, restaurants, attractions, and more by keywords or start URL. Enter check-in/out dates, and select currency and language. Promote locations, capture details, and retrieve emails and phone numbers if shared.
Getting "We got blocked" a LOT
Closed
12024-04-18T16:24:25.007Z INFO BasicCrawler: Starting the crawler. 22024-04-18T16:24:25.739Z WARN BasicCrawler: Reclaiming failed request back to the list or queue. RequestError: Stream closed with error code NGHTTP2_INTERNAL_ERROR 32024-04-18T16:24:25.741Z at file:///usr/src/app/src/main.js:51:19 {"id":"C3fNXPIwkGeg2je","url":"https://api.tripadvisor.com/api/internal/1.14/location/21342715?currency=USD&lang=en","retryCount":1} 42024-04-18T16:24:29.819Z INFO PHASE: -- Fetching place: https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html 52024-04-18T16:24:30.590Z WARN BasicCrawler: Reclaiming failed request back to the list or queue. We got blocked 62024-04-18T16:24:30.592Z at BasicCrawler.requestHandler (file:///usr/src/app/src/main.js:55:19) {"id":"BHiHD2BsASjaBmq","url":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html","retryCount":1} 72024-04-18T16:24:34.774Z WARN BasicCrawler: Reclaiming failed request back to the list or queue. We got blocked 82024-04-18T16:24:34.776Z at BasicCrawler.requestHandler (file:///usr/src/app/src/main.js:55:19) {"id":"BHiHD2BsASjaBmq","url":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html","retryCount":2} 92024-04-18T16:24:38.118Z WARN BasicCrawler: Reclaiming failed request back to the list or queue. We got blocked 102024-04-18T16:24:38.120Z at BasicCrawler.requestHandler (file:///usr/src/app/src/main.js:55:19) {"id":"BHiHD2BsASjaBmq","url":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html","retryCount":3} 112024-04-18T16:24:41.842Z WARN BasicCrawler: Reclaiming failed request back to the list or queue. We got blocked 122024-04-18T16:24:41.845Z at BasicCrawler.requestHandler (file:///usr/src/app/src/main.js:55:19) {"id":"BHiHD2BsASjaBmq","url":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html","retryCount":4} 132024-04-18T16:24:45.972Z WARN BasicCrawler: Reclaiming failed request back to the list or queue. We got blocked 142024-04-18T16:24:45.974Z at BasicCrawler.requestHandler (file:///usr/src/app/src/main.js:55:19) {"id":"BHiHD2BsASjaBmq","url":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html","retryCount":5} 152024-04-18T16:24:49.879Z ERROR BasicCrawler: Request failed and reached maximum retries. Error: We got blocked 162024-04-18T16:24:49.881Z at BasicCrawler.requestHandler (file:///usr/src/app/src/main.js:55:19) 172024-04-18T16:24:49.883Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 182024-04-18T16:24:49.885Z at async BasicCrawler._runRequestHandler (/usr/src/app/node_modules/@crawlee/basic/internals/basic-crawler.js:685:9) 192024-04-18T16:24:49.887Z at async wrap (/usr/src/app/node_modules/@apify/timeout/index.js:52:21) {"id":"BHiHD2BsASjaBmq","url":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html","method":"GET","uniqueKey":"https://www.tripadvisor.com/Hotel_Review-g189914-d21342715-Reviews-Vanha_Postitalo-Varkaus_Northern_Savonia.html"}
I can send many actor run URLs as an example, but I had 500 requests blocked yesterday...
Hey there!
Thank you very much for reaching out, using our actor, and letting us know about your problem. We checked the Run that you shared with us in detail and it seems like the problem is occurring due to the low qualified proxy IPs. By using the same queries with different proxy sets (or enriched), we retrieved the results without any problem.
At the current moment, you can kickstart the actor with the Residential Proxies to get better results. In the phase of Input, go to the Advanced Options and select Residential. Within that way, it should perform better.
Best
epctex
