No credit card required

Smart Article Extractor

lukaskrivka/article-extractor-smart

No credit card required

📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.

Back to issues Create new issue

How do i get article with certain keywords?

Closed

BSD_24 opened this issue

What do i need to modify in the actor inputs? Thanks!

Milan Knoll (milunnn)

Hi,

Could you please clarify what do you wish to accomplish? Do you already have sites from which to scrape, but just need a keyword filter for the articles?

BSD_24

Hi, sorry I just read your response. Suppose that I want to scrape from 2 sites, like newsweb.com and newsb.com. From each of these sites, I want to find articles only with certain keywords. For example in newsweb.com I want to get articles with "football" keyword. I tried filtering the raw data externally in python, but it still makes each scraping process highly expensive. Thanks

Ondrej Klinovský (ondrejklinovsky)

Hey,

unfortunately this actor can't do that. But you could use Google Search Results Scraper for that. See this example run: I used site: and intext: search operators to specify site and what word should the page contain: "site:cnn.com intext:football". You can then pass the results to this actor.

BSD_24

Hi, I have tried that and its working pretty good! Thanks for the info!

Add comment

Developer

Lukáš Křivka

Actor Metrics

308 monthly users
87 stars
>99% runs succeeded
15 days response time
Created in Nov 2019
Modified 2 months ago

Categories

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

6.3k

114

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

35.3k

981

Twitter Tweets and Profiles Scraper

web.harvester/twitter-scraper

Easily search and extract tweets from profiles or directly using a URL with our tool. Scrape tweets and replies effortlessly. Download data in formats like JSON, CSV, XML, RSS, or HTML Table, perfect for integration with various applications, databases, and social media analytics tools.

Web Harvester

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

53.6k

346

Google Trends Scraper

emastra/google-trends-scraper

Scrape data from Google Trends by search terms or URLs. Specify locations, define time ranges, select categories to get interest by subregion and over time, related queries and topics, and more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Emiliano Mastragostino

Easy Twitter Search Scraper

web.harvester/easy-twitter-search-scraper

Easily Scrape tweets with our Twitter Search Scraper. Export data in formats like JSON and Excel perfect for integration with various applications, databases, and data analysis tools.

Web Harvester

Reddit Scraper Lite

trudax/reddit-scraper-lite

Pay Per Result, unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.

Gustavo Rudiger

4.6k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.