Smart Article Extractor avatar
Smart Article Extractor
Try for free

No credit card required

View all Actors
Smart Article Extractor

Smart Article Extractor

lukaskrivka/article-extractor-smart
Try for free

No credit card required

📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.

User avatar

Why isn't it scraping articles from the following website?

Closed

ybierens opened this issue
10 months ago
User avatar

Patai5

10 months ago

Hi Yannick, It seems like the articles are loaded dynamically, try setting useBrowser to true. Let me know if it helps. Thanks!

User avatar

ybierens

10 months ago

Thank you, now it does work! Would it be wise to always setting useBrowser to true? Are there any drawbacks except for it being expensive?

User avatar

Patai5

10 months ago

Awesome, glad to hear :D Browser search is slower compared to non-browser search, which makes it more expensive. There are no other drawbacks, only advantages like the ability to scrape dynamic web pages (your case), at the sacrifice of speed. Happy scraping!

User avatar

You can set it to true only if the initial scrape will not work. That way you first scrape all sites with the cheaper method and only fallback to browser

User avatar

ybierens

10 months ago

Hey! Thank you for your reply! Where can I find that option? I don't see it under the 'browser options' section.

User avatar

It is there 'Use Browser (Puppeteer)'

User avatar

ybierens

10 months ago

Ah i see, I thought you had to specifically set it as a fallback, but as I understand it now that's how it always works then.

Developer
Maintained by Apify
Actor metrics
  • 185 monthly users
  • 100.0% runs succeeded
  • 2.8 days response time
  • Created in Nov 2019
  • Modified about 2 months ago
Categories