No credit card required
Smart Article Extractor
No credit card required
📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.
Why isn't it scraping articles from the following website?
Closed
https://www.ggn.nl/over-ggn/nieuws-kennis/
I tried it multiple times
Patai5
Hi Yannick,
It seems like the articles are loaded dynamically, try setting useBrowser to true.
Let me know if it helps. Thanks!
ybierens
Thank you, now it does work! Would it be wise to always setting useBrowser to true? Are there any drawbacks except for it being expensive?
Patai5
Awesome, glad to hear :D Browser search is slower compared to non-browser search, which makes it more expensive. There are no other drawbacks, only advantages like the ability to scrape dynamic web pages (your case), at the sacrifice of speed. Happy scraping!
You can set it to true only if the initial scrape will not work. That way you first scrape all sites with the cheaper method and only fallback to browser
ybierens
Hey! Thank you for your reply! Where can I find that option? I don't see it under the 'browser options' section.
It is there 'Use Browser (Puppeteer)'
ybierens
Ah i see, I thought you had to specifically set it as a fallback, but as I understand it now that's how it always works then.
Lukáš Křivka- 194 monthly users
- 47 stars
- 99.9% runs succeeded
- 1.9 days response time
- Created in Nov 2019
- Modified 15 days ago
