No credit card required

Smart Article Extractor

lukaskrivka/article-extractor-smart

No credit card required

📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.

Do you want to learn more about this Actor?

Get a demo

Back to issues Create new issue

Why isn't it scraping articles from the following website?

Closed

ybierens opened this issue

https://www.ggn.nl/over-ggn/nieuws-kennis/

I tried it multiple times

Patai5

Hi Yannick, It seems like the articles are loaded dynamically, try setting useBrowser to true. Let me know if it helps. Thanks!

ybierens

Thank you, now it does work! Would it be wise to always setting useBrowser to true? Are there any drawbacks except for it being expensive?

Patai5

Awesome, glad to hear :D Browser search is slower compared to non-browser search, which makes it more expensive. There are no other drawbacks, only advantages like the ability to scrape dynamic web pages (your case), at the sacrifice of speed. Happy scraping!

Lukáš Křivka (lukaskrivka)

You can set it to true only if the initial scrape will not work. That way you first scrape all sites with the cheaper method and only fallback to browser

ybierens

Hey! Thank you for your reply! Where can I find that option? I don't see it under the 'browser options' section.

Lukáš Křivka (lukaskrivka)

It is there 'Use Browser (Puppeteer)'

ybierens

Ah i see, I thought you had to specifically set it as a fallback, but as I understand it now that's how it always works then.

Add comment

Developer

Lukáš Křivka

Actor Metrics

197 monthly users
65 stars
>99% runs succeeded
1.2 days response time
Created in Nov 2019
Modified 4 months ago

Categories

News

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

26.9k

663

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

49.7k

234

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

3.7k

Twitter Tweets and Profiles Scraper

web.harvester/twitter-scraper

Easily search and extract tweets from profiles or directly using a URL with our tool. Scrape tweets and replies effortlessly. Download data in formats like JSON, CSV, XML, RSS, or HTML Table, perfect for integration with various applications, databases, and social media analytics tools.

Web Harvester

2.3k

Reddit Scraper Lite

trudax/reddit-scraper-lite

Pay Per Result, unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.

Gustavo Rudiger

3.8k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

1.2k

Dun & Bradstreet Scraper

epctex/dnb-scraper

Effortlessly extract valuable company information, financial projections, industry insights, and more from the extensive Dun & Bradstreet commercial database. Dive deep into the D&B Data Cloud, Business Directory, articles, companies, and industries with customized search terms.

epctex

1.7k

Google News Scraper

lhotanova/google-news-scraper

Gets featured articles from Google News with title, link, source, publication date and image.

Kristýna Lhoťanová

982

TikTok Comment API

novi/tiktok-comment-api

Unlock the power of a blazing fast TikTok Comment API. Seamlessly retrieve comments with speed and efficiency. Optimize your TikTok data analysis with this powerful tool.