Web Scraper
No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.
How do I setup pagination with a URL
I have a URL that I need to extract data from, but there are 75+ pages to click through. How can I set that up in the input section?
Hello sacdrexelmba,
Thank you for your question! Since the pages on the URL you provided do not have separate URLs, here are two suggestions to set up pagination:
-
Click Pagination in a Loop
Use the Page Function in your Actor to click through the pagination buttons (and store the data) in a loop until all pages are processed. -
Inspect API Requests : The page makes API requests as you paginate, such as:
https://www.hlth.com/api/hlth-id?eventId=9f173c20-7f93-44ce-847a-8396562d4165&page=1&limit=100. Thepagequery parameter changes with each page. You can scrape data faster and more efficiently by targeting this API directly using tools like our Cheerio Scraper, which doesn't require a browser.
Since these approaches should resolve the issue, I’ll close this ticket. If you need further assistance, feel free to reopen or start a new thread. Cheers!
ApifyActor Metrics
3.3k monthly users
-
456 bookmarks
>99% runs succeeded
4.8 days response time
Created in Mar 2019
Modified a month ago
