No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.
Shopee scraper problem
Closed
I'm trying to use the scraper to extract information about searching for items in the store, but without success. After some research, I noticed that the information on the page only loads after a few seconds. Any tips on how to resolve this?
URL example: https://shopee.com.br/search?keyword=chinelo
Hello and thank you for your interest in this Actor!
Usually with dynamic-loaded websites, it should be enough to set the Performace and Limits > Navigations waits until to networkidle0 - this way, the scraper should wait until there are no active (asynchronous) requests between the client and the server.
Unfortunately, there seems to be some issue with the Actor itself - even locally, the website just doesn't load properly. We already have an idea about what might be the culprit, but we'll need to run some tests first to be absolutely sure.
I'll keep you posted about any updates regarding this issue. Thank you! (and sorry for the inconvenience).
lojahub
Hi Jindřich Bär, first of all thank you for the message and interest helping. I'm waiting for news. Thank you !
lojahub
Hello, Jindřich Bär hoping you're doing great, do you have any news regarding the problem ?
Hello and sorry for the delay.
Upon closer examination, the Actor seems to be working just fine. The networkidle0 / networkidle2 option might not be the best choice for this website, as there might be some long-running HTTP requests (think polling the API for new items etc.), which cause the Actor to never stop loading. When you switch the Performace and Limits > Navigations waits until option to load, it seems to load the page alright - check out my example run.
It's possible that over the past 2 months, the web site might have changed (so the original issues aren't there anymore).
I'll close this issue now, but feel free to reopen it (or open a new one), if you encounter any other issue with this Actor.
Again, sorry for the wait, and thank you for your patience. Cheers!
Apify- 2.3k monthly users
- 119 stars
- 99.9% runs succeeded
- 5.2 days response time
- Created in Mar 2019
- Modified about 1 month ago
