Web Scraper
No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.
If I verify in pageFunction that the target website has been redirected by the anti-crawl mechanism, how do I get the latest page content.
If I verify in pageFunction that the target website has been redirected by the anti-crawl mechanism, how do I get the latest page content.
Hello and thank you for your interest in this Actor!
Pages like AliExpress invest heavily in anti-scraping protection, and the tricks available in Web Scraper might not always be enough. While you can try workarounds such as switching to Residential proxies or enabling media downloads, AliExpress may still recognize Web Scraper’s network fingerprints.
On the Apify Store, you can find multiple third-party Actors specifically designed for AliExpress scraping. These might be a better fit for your use case. Alternatively, you can develop your own custom solution and deploy it on Apify for more flexibility.
I'll go ahead and close this issue, but feel free to open a new one if you need further assistance!
ApifyActor Metrics
3.3k monthly users
-
456 bookmarks
>99% runs succeeded
4.8 days response time
Created in Mar 2019
Modified a month ago
