Playwright Scraper
Pricing
Pay per usage
Playwright Scraper
Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.
4.3 (7)
Pricing
Pay per usage
31
Monthly users
141
Runs succeeded
98%
Response time
3.1 days
Last modified
10 months ago
I don't get the list content of the website
The content is likely dynamically loaded by JS but I don't get the table and list items returned. Here is the website:
Hello max-sfr,
Thank you for reaching out. It appears that the content you’re trying to scrape is dynamically loaded by JavaScript. To handle this, you can:
- Use Playwright's
waitForSelectororwaitForTimeoutmethods to ensure specific elements on the page have loaded before proceeding with scraping.
For example:1await page.waitForSelector('selector'); // Wait for a specific element to appear 2await page.waitForTimeout(5000); // Wait for a fixed time to allow the page to load - Inspect the API requests the page is making. The page sends POST requests to:
https://www.dtvp.de/Center/api/v2/project/search
The response is in JSON format, which you can scrape more efficiently without rendering the page in a browser. You can use e.g. our Cheerio Scraper to scrape the API directly.
Since this should address your issue, I’ll be closing this ticket. If you have any more questions, feel free to ask. Cheers!
Pricing
Pricing model
Pay per usageThis Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.
