Web Scraper
Pricing
Pay per usage
Web Scraper
Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.
4.4 (22)
Pricing
Pay per usage
762
Total users
85K
Monthly users
4.5K
Runs succeeded
>99%
Issues response
31 days
Last modified
15 hours ago
can't extract emails from websites doing dymanic content rendering
Closed
Failed multiple times to extract emails from url https://www.rovic.com/ multiple times..
Hello, and thank you for your interest in this Actor!
I tried checking a few Web Scraper runs under your account, and while I'm not 100% this is the issue, I might have an idea what might be going on.
In one of the recent runs, you have the following expression in the Page Function:
Note that Web Scraper runs the pageFunction inside the browser (as if you ran it in your browser console). Because of this, you don't need to call context.page.evaluate (context.page is actually undefined), and you can simply assign const bodyText = document.body.innerText instead.
If you want to access the page object specifically, you can do so in our Playwright Scraper or Puppeteer Scraper. Both those Actors run your custom function in the Node.JS process and pass the page object there as a parameter.
I'll close this issue now, but feel free to ask additional questions if you have any. Cheers!
