Web Scraper
Pricing
Pay per usage
Web Scraper
Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.
4.5 (22)
Pricing
Pay per usage
651
Monthly users
3.7k
Runs succeeded
>99%
Response time
25 days
Last modified
6 days ago
can't extract emails from websites doing dymanic content rendering
Closed
Failed multiple times to extract emails from url https://www.rovic.com/ multiple times..
Hello, and thank you for your interest in this Actor!
I tried checking a few Web Scraper runs under your account, and while I'm not 100% this is the issue, I might have an idea what might be going on.
In one of the recent runs, you have the following expression in the Page Function:
1const { page, request, log } = context; 2 3const bodyText = await page.evaluate(() => document.body.innerText);
Note that Web Scraper runs the pageFunction inside the browser (as if you ran it in your browser console). Because of this, you don't need to call context.page.evaluate (context.page is actually undefined), and you can simply assign const bodyText = document.body.innerText instead.
If you want to access the page object specifically, you can do so in our Playwright Scraper or Puppeteer Scraper. Both those Actors run your custom function in the Node.JS process and pass the page object there as a parameter.
I'll close this issue now, but feel free to ask additional questions if you have any. Cheers!
Pricing
Pricing model
Pay per usageThis Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.
