Google News Keyword & Full-Text Article Scraper

Scrape Google News without a login, API key, or browser. Search by keyword, browse a topic, convert a Google News link, read any RSS/Atom feed, or send publisher article URLs for full-text extraction. Every record carries a clean headline, the real publisher URL instead of a Google redirect, a canonical-URL dedupe key, and an honest extraction status when a publisher refuses the page.

What this actor does

• Five input modes: keyword, topic, newsUrl, rss, and articleUrls.
• Real publisher links: Google News article tokens are resolved in batches, with the Google link kept as a fallback if the token ever changes.
• Date-correct results: every item is filtered against its actual publishedAt time, so a "past day" run cannot leak a newer or older article. In keyword mode, multi-day ranges are walked day by day, which also gets past Google's ~100-item feed cap; topic and RSS modes return the feed's available items.
• Clean, deduplicated records: the - Publisher suffix is removed from titles, and the same story is deduplicated by canonical publisher URL across queries and day windows.
• Optional, honest full text: page text, description, author, and image are extracted only when requested (always in articleUrls mode). A blocked, paywalled, timed-out, or short page becomes a record with fullTextStatus instead of a silent null.
• Empty fields are omitted from every record.

Output per article

• title — headline with Google's publisher suffix removed
• url — the publisher's article URL; the Google News link when resolution fails
• source — publisher name from the feed or page
• domain — publisher hostname, e.g. bbc.com
• publishedAt — ISO 8601 UTC publication time from the feed
• snippet — publisher description when a page was fetched; omitted for metadata-only runs because Google's RSS "description" only repeats the title
• author — byline from page metadata when available
• imageUrl — article image from the publisher page or feed
• language — edition language used for the run
• fullText — extracted article body when full text was requested and available
• fullTextStatus — ok, short, empty, blocked, paywalled, rate_limited, not_found, timeout, error, unresolved, or skipped_cap
• query — the keyword, topic, feed URL, or article URL that produced the row
• recordType — always article
• sourceUrl — the Google News or feed link the record came from
• scrapedAt — UTC timestamp when the run assembled the record

Input

Field	Type	Default	Description
mode	string	keyword	keyword, topic, newsUrl, rss, or articleUrls
queries	array	–	Keywords for keyword mode; Google operators such as site: and OR work
topics	array	–	Topic names (WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH), TOP, topic hashes, or Google News topic URLs (topic mode)
newsUrls	array	–	Google News search/topic URLs or RSS/Atom feeds (newsUrl mode)
rssUrls	array	–	Any public RSS or Atom feed (rss mode)
articleUrls	array	–	Publisher or Google News article URLs (articleUrls mode); full text is always attempted
language	string	en	Edition language, e.g. en, de, ja, hi
country	string	US	Edition country, e.g. US, GB, DE, IN
dateRange	string	any	any, 1h, 6h, 12h, 1d, 7d, 30d, 1y, or custom
startDate	string	–	Custom range start, YYYY-MM-DD, inclusive
endDate	string	–	Custom range end, YYYY-MM-DD, inclusive
maxResultsPerQuery	integer	20	Records taken from each feed request (1–100, Google's feed cap is about 100)
maxItems	integer	500	Hard cap on unique records in the run (1–5000)
extractFullText	boolean	false	Fetch publisher pages for body text, description, author, and image
maxFullText	integer	100	Maximum pages fetched per run (1–500); later records get fullTextStatus=skipped_cap
proxyConfiguration	object	off	Optional; no proxy is needed or recommended by default

Dates and caps

• Relative ranges use Google's recency operator and are then post-filtered against the real publication time.
• A custom startDate/endDate is inclusive. Items published after the end are always dropped, even if Google's search operator returns them.
• When a date range spans several days and you ask for more than one feed page, the actor walks day windows (newest first) and collects up to maxItems. Full windows are reported with a truncation signal in the run status.
• A single Google News feed returns roughly 100 items. maxResultsPerQuery caps each feed window; maxItems caps the whole run.
• Full text is capped per run so a large metadata pull cannot turn into thousands of publisher requests.

Examples

Keyword scan, fastest mode

{
  "mode": "keyword",
  "queries": ["artificial intelligence", "semiconductor policy"],
  "dateRange": "7d",
  "maxResultsPerQuery": 20,
  "maxItems": 100
}

Topic feed with full text

{
  "mode": "topic",
  "topics": ["TECHNOLOGY", "SCIENCE"],
  "extractFullText": true,
  "maxFullText": 50,
  "maxItems": 50
}

Custom date range and real publisher URLs

{
  "mode": "keyword",
  "queries": ["central bank interest rates"],
  "dateRange": "custom",
  "startDate": "2026-10-01",
  "endDate": "2026-10-20",
  "maxResultsPerQuery": 100,
  "maxItems": 500
}

Backfill known article URLs

{
  "mode": "articleUrls",
  "articleUrls": [
    "https://example.com/news/story-one",
    "https://example.com/news/story-two"
  ],
  "maxFullText": 20
}

Read an existing RSS feed

{
  "mode": "rss",
  "rssUrls": ["https://feeds.bbci.co.uk/news/rss.xml"],
  "maxResultsPerQuery": 30,
  "maxItems": 30
}

Use cases

• Media monitoring — track brand, competitor, or industry coverage across thousands of publishers.
• Newsletter and digest curation — pull a dated, deduplicated story list for a daily or weekly issue.
• NLP and RAG corpora — collect headlines now and full article bodies for the subset that matters.
• Market and event research — build a date-bounded dataset around an earnings call, election, or product launch.
• PR and communications reporting — measure coverage volume and collect the underlying links.
• Academic and policy research — export a reproducible, date-correct news sample with publisher URLs.

FAQ

Does this return the real article URL or a Google redirect?

The real publisher URL. Google News tokens are resolved through Google's own article endpoint. If Google changes the token format, the actor keeps the Google News link rather than losing the record, and says so in the run status.

Why is snippet missing on a metadata-only run?

Google News RSS does not contain a real summary — its description is just the headline plus the publisher name. Returning that as a snippet would duplicate title. A real snippet comes from the publisher page, so it appears only when extractFullText is on or in articleUrls mode.

What do the fullTextStatus values mean?

ok means a substantial body was extracted; short means a small body was found; empty means no body; blocked, rate_limited, not_found, timeout, and error describe publisher responses; paywalled means a subscription wall was detected; unresolved means the publisher URL could not be resolved; skipped_cap means the record was beyond maxFullText.

Can full text read paywalled articles?

No. The actor reads only what a publisher serves publicly. It does not bypass paywalls, logins, or consent walls, and it never claims 100% full-text coverage. Premium publishers may return a short preview or a blocked status.

Why did I get fewer than 100 results for one query?

Google News returns roughly 100 items per feed request and has no working pagination parameter. The actor splits multi-day ranges into day windows to collect more, up to maxItems. If a window hits the feed cap, the run reports a truncation signal.

How does the date filter work?

Google's own date operators are day-granular and timezone-fuzzy. The actor adds the operator to the search, then drops every item whose real publishedAt falls outside your window. That is why a "past day" run does not include tomorrow's articles or yesterday's late items.

Are results deduplicated?

Yes. The same story is deduplicated by canonical publisher URL after resolution, with a Google article ID and a title/source key as fallbacks. Tracking parameters, www., fragments, and trailing slashes do not create duplicate rows.

Does a blocked publisher stop the run?

No. A 401, 403, 429, timeout, or paywall produces a normal record with the RSS metadata and a fullTextStatus. The rest of the run continues. If the whole run produces zero records, the actor fails loudly with the feed errors instead of pretending it succeeded.

Do I need a proxy or an API key?

No. Google News RSS and publisher pages are reached directly. A proxy is optional and usually unnecessary; residential proxies can actually make Google rate-limit more.

Is this affiliated with Google?

No. This is an independent, third-party actor that reads publicly available Google News feeds and publisher pages. It is not affiliated with or endorsed by Google. You are responsible for respecting publisher terms when storing or redistributing article text.