News Scraper - Keyword News Search & Monitoring API

What does News Scraper do?

Enter keywords (and optionally countries, languages, topics or your own sites), get one row per news article with headline, link, publisher, time and the publisher's own description and keywords, at $1 per 1,000 articles. Article page details cost $0.50 per 1,000 and are optional. Filtered-out articles, duplicate stories, articles returned by earlier runs, unreadable sources and invalid inputs are free. No start fee.

It searches the latest news of 250+ publishers by keyword in one run and returns one clean row per article: headline, link, publisher, publication time, country, language, the searches it matched, plus the description, news keywords, categories, author and image the publisher provides.

• 🔎 Keyword search with a real query language: all words must appear, "exact phrase", -exclude, OR, prefix*; several searches per run, each article lists which ones it matched.
• 🌍 254 publishers in 47 countries and 22 languages (Reuters-style wires excepted, see Limitations): BBC, CNN, The New York Times, Bloomberg, Forbes, TechCrunch, The Verge, Der Spiegel, Le Monde, El País, NHK, The Times of India, Straits Times, 中央社 and many more. Filter by country, language and topic (technology, business, finance, crypto, science, health, sports, press releases...).
• ➕ Your own sources: paste any site, section page, RSS / Atom feed or Google News sitemap (trade press, company newsrooms, local papers, blogs); feeds are found automatically.
• 🧹 Duplicates collapsed: the same wire story at ten publishers comes back once, earliest copy first, with the others in alsoPublishedBy (free).
• 🔔 Only new articles for schedules: articles returned by an earlier run of the same monitor are skipped for free.
• 📰 Optional article details: authors, section, tags, modified time, canonical link, paywall flag and word count from the article page's own metadata (no article text).
• 📄 CSV file for Excel or Google Sheets, and a SOURCES record showing what every publisher returned.

Why this Actor?

• No search-engine scraping, so no "0 results" runs. Google News scrapers break when Google changes its pages or blocks them. This Actor reads what publishers publish for crawlers and readers: their Google News sitemaps (every article of the last ~48 hours with headline, time and keywords) and their RSS / Atom feeds. In our tests all 254 publishers were read in every run.
• Fresh and complete per publisher. A news sitemap lists every article a publisher put out in the last two days, not just the 10-100 items a search page shows.
• Honest crawler. Every request names itself (TidyToolsBot), follows each site's robots.txt and Content-Signal rules, and asks one thing at a time per site. Nothing is bypassed.
• Cheap and predictable: $1 per 1,000 articles, no start fee. Filtered articles, collapsed duplicates, already-seen articles and unreadable sources are free.

Real results (3-4 October 2026, run on our machine)

• OpenAI OR Anthropic and "electric vehicle" battery -recall, English: 165 English-language publishers read, 42,723 articles checked, 135 matched, 50 returned in 32 seconds for $0.05 (Business Insider, Forbes, TechCrunch, Engadget, Fox News, Yahoo Finance, Decrypt...).
• Germany, Austria, Switzerland: Merz OR Bundesregierung, Klimaschutz, sorted by relevance, details for the first 15: 15 publishers, 3,532 articles checked, 40 returned + 15 article details in 6.5 seconds for $0.0475 (WELT, BILD, Der Spiegel, n-tv, FAZ, SZ, Handelsblatt, tagesschau).
• All 254 publishers at once (two searches, no filters): 67,568 articles checked in 47 seconds.
• Your own sources (Cloudflare blog, simonwillison.net, an OpenAI RSS link, TechCrunch, plus 4 bad lines), last 7 days: 30 articles for $0.03 in 3.3 seconds; The Guardian (HTTP 403), a site without a feed, a missing sitemap and a non-URL came back as 4 free error rows.
• Only new: Nvidia OR AMD OR Intel monitor, first run 90 articles ($0.09); second run a minute later 0 articles, $0.00 (90 returned before).

Use cases

• Media monitoring and PR: your brand, products, executives or competitors across major publishers, every few hours into Slack, email or a sheet.
• Market and investment research: news about a company, sector or commodity by country and language.
• AI agents and RAG: a fresh, structured news feed with links and descriptions; give the links to a page reader when you need the full text.
• Trade press tracking: add your industry's sites and newsrooms as sources and get only new articles.
• Content and SEO teams: what the press is writing about a topic right now, with the publishers' own keywords.

How to use it

1. Type one or more keywords (one search per line), e.g. OpenAI OR Anthropic, "interest rates" -mortgage, tariff*.
2. Optional: pick countries, languages and topics, add your own sources, and set published within (hours) and maximum articles.
3. For alerts, turn on Only new articles and schedule the Actor (every 3-6 hours is enough: news sitemaps keep about 48 hours).
4. Run. Results are in the dataset (views: Articles, Keywords and duplicates, Article page details, Errors), in ARTICLES.csv and in SOURCES.

Input example

{
    "keywords": ["OpenAI OR Anthropic", "\"electric vehicle\" battery -recall"],
    "languages": ["en"],
    "maxAgeHours": 48,
    "maxArticles": 50,
    "sortBy": "newest"
}

Monitoring on a schedule: {"keywords": ["Acme Corp", "\"Jane Doe\" Acme"], "onlyNew": true, "monitorName": "acme-press"}. Only your sources: {"sources": ["https://blog.cloudflare.com", "simonwillison.net", "https://openai.com/news/rss.xml"], "onlyMySources": true, "maxAgeHours": 168}.

Output example

One article from a real run (3 October 2026):

{
    "type": "article",
    "rank": 3,
    "title": "Former Anthropic security leader warns AI agents are becoming too autonomous for humans to keep them in check",
    "url": "https://www.foxnews.com/tech/former-anthropic-security-leader-warns-ai-agents-becoming-too-autonomous-humans-keep-check",
    "publisher": "Fox News",
    "sourceDomain": "foxnews.com",
    "publishedAt": "2026-10-03T17:26:10.000Z",
    "country": "US",
    "language": "en",
    "topics": ["general", "politics"],
    "matchedQueries": ["OpenAI OR Anthropic"],
    "matchedInTitle": true,
    "description": "Jeffrey Ladish, a former Anthropic security team member, warns AI agents are colluding and hacking with no reliable solution to keep them in check.",
    "keywords": [],
    "categories": ["Fox News", "fox-news/tech/artificial-intelligence", "fox-news/politics", "fox-news/tech"],
    "author": null,
    "imageUrl": "https://a57.foxnews.com/static.foxnews.com/foxnews.com/content/uploads/2026/10/931/523/jeffrey-ladish.jpg?ve=1&tl=1",
    "foundVia": ["news-sitemap", "feed"],
    "alsoPublishedBy": [],
    "success": true,
    "charged": true,
    "scrapedAt": "2026-10-03T19:02:11.000Z"
}

With Add article page details, a row also gets (shortened, from the German run):

"details": {
    "canonicalUrl": "https://www.bild.de/politik/clara-von-nathusius-friedrich-merz-wird-keine-wahlen-mehr-gewinnen-6abf9a17863d7086e22737e0",
    "metaDescription": "CDU-Influencerin Clara von Nathusius kritisiert Merz, Parteistrukturen und die Koalition. ...",
    "authors": [],
    "tags": ["CDU", "Merz Friedrich"],
    "modifiedAt": "2026-10-03T14:20:00.000Z",
    "isAccessibleForFree": null,
    "wordCount": null
}

What the fields mean

Field	Meaning
title, url, publisher, sourceDomain	Headline, article link (tracking parameters removed), publisher name and the site it was read from.
publishedAt	Publication time (UTC) from the news sitemap or feed; the earliest when both give one.
country, language, topics	Publisher country and topics; article language (from the news sitemap when it says so, else the publisher's).
matchedQueries, matchedInTitle	Which of your searches matched, and whether the match is in the headline.
description, keywords, categories, author, imageUrl	What the feed / news sitemap gives: summary (up to 600 characters), the publisher's news keywords, feed categories, author, image. Empty when the publisher does not provide it.
foundVia	news-sitemap and/or feed.
alsoPublishedBy	Other publishers with the same headline (collapsed duplicates): publisher, link, time.
details, detailsError	Optional article page metadata (see above) or why it could not be read (free).
isNew	Only-new mode: always true on returned rows (older ones are skipped).
rank, success, charged, errorType, error	Position in the results and the usual TidyTools status fields.

How much does it cost?

Event	Price	When
Article	$1.00 / 1,000 ($0.001 each)	one article returned after your filters
Article details	$0.50 / 1,000	only with "Add article page details", and only when the page published metadata

Free: articles dropped by your filters, duplicate copies of a story, articles returned before in only-new mode, sources that cannot be read, invalid inputs, details of blocked or empty pages. No start fee. Silver plans get 10% off, Gold and above 20%.

For comparison (Apify Store, checked October 2026): data_xplorer/google-news-scraper-fast $4.00 / 1,000 results, easyapi/google-news-scraper $5.00, automation-lab/google-news-scraper $2.30, crawlerbros/google-news-scraper $2.00.

Control your cost

• Maximum articles caps each run; a run checks tens of thousands of articles but you only pay for those returned.
• Apify's maximum charge per run is respected: the run saves what fits and lists the rest in SUMMARY.notProcessed.
• The plan (worst case) is printed at the start of the run and stored in SUMMARY.costPlan.

Use with AI agents (MCP)

Connect Apify's MCP server (https://mcp.apify.com?tools=tidytools/news-scraper) to Claude, Cursor or any MCP client, then ask e.g. "Find today's news about Nvidia export rules in English and German and give me the links." Minimal input:

{ "keywords": ["Nvidia export"], "languages": ["en", "de"], "maxArticles": 30 }

Failed sources and inputs are not charged and carry an errorType; SUMMARY.status is SUCCESS, PARTIAL_RESULTS, NO_RESULTS, LIMIT_REACHED or FAILED.

Limitations

• Coverage is the publisher list plus your sources, not the whole web like Google News. Some large publishers are not included because their robots.txt closes their feeds and sitemaps to crawlers, or they refused our requests when the list was built (Reuters, AP, The Guardian, USA Today and about 50 others). You can still add any site as a source; if it refuses, you get a free error row.
• Recency: news sitemaps hold about 48 hours, feeds the latest items. For a history, schedule the Actor with only-new mode; it cannot search past months.
• Headline-level data: description, keywords and author depend on what each publisher puts in its feed or sitemap (some give only the headline). No article text is returned: it is the publishers' copyrighted content. Use a page reader on the links if you need the text for your own analysis.
• Keywords match whole words (no stemming): tariff does not match tariffs; use tariff*. Chinese, Japanese and Thai keywords match as substrings.
• Articles whose feed gives no date (or an impossible one) are skipped unless you turn on "Include articles without a date".
• Publisher websites can change their feeds; the built-in list is re-checked regularly and SOURCES shows any publisher that failed in your run (free).

FAQ

Does the Actor collect usage data? The Actor sends one anonymous usage counter per run (input type, item count range, outcome) to improve the tool; no URLs, content or account data.

What happens if my input is wrong? The run ends normally (it does not fail) with a free row, errorType: "invalid_input", and a message saying what to fix: a search that only excludes words, an unknown topic, "only my sources" with no sources, or country / language codes that match no publisher (countries are ISO codes such as GB, languages two-letter codes such as en). A source line that is not an address, a site that cannot be reached or one without any feed is a free error row for that line; the other sources still run. Nothing is charged for any of them. A run where no article matches leaves one free no_results row naming the filter that dropped the most articles (none in "Only new" runs that simply have nothing new).

Is this a Google News scraper? No. It does not touch Google. It reads the same publisher feeds and news sitemaps that news aggregators read, so it keeps working when search pages change.

How fresh are the results? As fresh as the publishers' own sitemaps and feeds, usually minutes after publication.

Can I get the full article text? No. The Actor returns headlines, links and the metadata publishers make available. Respecting copyright, the article body is not copied.

Why did a publisher return nothing? See the SOURCES record in the run's key-value store: it lists each source, the feeds and sitemaps read, how many articles they had and any error.

Can I add paywalled sites? Yes, if their feeds or news sitemaps are public (headline, link and time are returned; the article text never is). Article details include isAccessibleForFree when the page declares it. Some paywalled publishers refuse article-page requests (HTTP 403): those rows get a detailsError and the details are free.

Can I filter to one publisher? Add its domain under "Your own sources" and turn on "Only my sources" (for a publisher in the built-in list its verified feeds are used), or use countries, languages and topics to narrow the list. Use "Exclude domains" to drop publishers.

Is it legal? It only reads public feeds and sitemaps that publishers offer for this purpose, follows robots.txt, identifies itself and returns metadata and links, not article text. Check the terms of the sites you add yourself.