Google News Sitemap Extractor

Fetch any public Google News sitemap (news: namespace) and emit per-article rows: title, publication date, publication name+language, genres, keywords, and stock tickers. Handles sitemap indexes (BFS, deduped). Sits next to the public sitemap-delta-monitor as a news-only pre-filter for media monitoring and RAG news feeds.

What it does

• Reads a public HTTP(S) URL of a Google News sitemap or a sitemap index that points to one.
• Walks sitemap indexes breadth-first, deduplicating by URL.
• Emits one dataset row per article that matches an optional from / to date window.
• Drops articles missing <news:publication_date> unless includeUnparsedDate is true.
• Records indexes / skipped non-news sitemaps / fetch errors as separate row kinds so the run is auditable.
• Pure data utility. Public sitemaps only — no auth, no anti-bot bypass, no scraping of article bodies.

Who it is for

• Media monitoring / news aggregators that need a fresh window of publisher articles.
• RAG pipelines that want a fast, structured feed of news titles + publication dates for re-indexing.
• Trading / equities workflows that mine stockTickers from publisher feeds.
• SEO teams that want to confirm a publisher's Google News sitemap is being kept fresh and well-formed.
• Anyone who needs a deterministic, low-cost pull from the public news: namespace without standing up their own crawler.

Quick start

1. Pick a publisher's Google News sitemap URL. The schema is documented at https://www.google.com/schemas/sitemap-news/0.9.

2. Run with the minimum input — sitemapUrl only:

   {
     "sitemapUrl": "https://www.example.com/news-sitemap.xml"
   }
   

3. Read the dataset. Each row has _kind: "article" and fields url, title, publicationDate, publication: { name, language }, genres[], keywords[], stockTickers[], plus _source (which sitemap it came from).

Date window

{
  "sitemapUrl": "https://www.example.com/news-sitemap.xml",
  "from": "2025-01-01",
  "to": "2025-12-31",
  "maxArticles": 5000
}

from and to accept YYYY-MM-DD or full ISO 8601 timestamps. Both bounds are inclusive.

Full input

Field	Type	Default	Notes
sitemapUrl	string (required)	—	Public HTTP(S) URL of a news sitemap or sitemap index
from	string	—	ISO date or timestamp, inclusive lower bound
to	string	—	ISO date or timestamp, inclusive upper bound
includeUnparsedDate	bool	false	If true, keep articles whose <news:publication_date> cannot be parsed
maxArticles	int	5000	Cap on emitted article rows
timeoutSec	int	20	Per-request HTTP timeout in seconds
maxBytes	int	4000000	Truncate sitemap body at this many bytes

Output

Dataset row kinds:

• article — one per emitted article. Carries url, title, publicationDate, publication.name, publication.language, genres[], keywords[], stockTickers[], _source.
• index — sitemap index discovered, with childCount and children[].
• skipped — sitemap without a news:news namespace, with a short reason.
• error — failed fetch, with HTTP status and error message.

SUMMARY key in key-value store:

{
  "ok": true,
  "root": "https://www.example.com/news-sitemap.xml",
  "articles": 132,
  "visited": 4,
  "childCounts": [
    { "url": "https://www.example.com/news-sitemap-1.xml", "articles": 70, "inWindow": 50 },
    { "url": "https://www.example.com/news-sitemap-2.xml", "articles": 90, "inWindow": 82 }
  ]
}

Pricing

This Actor is Pay Per Event on the Apify Store:

• apify-actor-start — flat per run.
• article_extracted — charged once per emitted article row that matches the date window.

Pricing details and per-unit USD prices are visible on the Store page. For local tests, Actor.charge is a no-op (no usage billed, no payout).

Limits

• Designed for public Google News sitemaps. Sites that require login, cookies, or bot bypass are out of scope.
• maxArticles is a hard cap on emitted rows; if you hit it, raise the cap and re-run.
• maxBytes truncates the body of a single sitemap. Bump it for very large news sitemaps.
• Sitemap indexes are followed with a simple BFS; very deep index chains (>50 levels) are not supported.

FAQ

Q: Is this a scraper of article pages? A: No. It only reads public sitemap XML. Article bodies are never fetched.

Q: My publisher doesn't publish a Google News sitemap — what now? A: This Actor will record _kind: "skipped" for that URL. Use the public sitemap-delta-monitor for generic sitemaps or product-feed-delta-monitor for product/feed use cases.

Q: Why is my run returning 0 articles? A: Check the date window — most news sitemaps are deeply time-bounded. Try a wider from / to or remove it entirely. Also verify <news:publication_date> is well-formed; unparseable dates are dropped unless includeUnparsedDate: true.

Q: Can I run it on a private sitemap? A: Only if the Actor host can reach it over plain HTTP(S) without auth. The Actor does not handle cookies, headers, login, or anti-bot challenges.

Q: How does this differ from a generic XML feed parser? A: It understands the news: namespace and pulls publication meta, genres, keywords, and stock tickers. For arbitrary RSS/Atom/JSON feeds use xml-feed-parser or rss-feed-delta-monitor.

Related Actors

• sitemap-delta-monitor — generic sitemap URL/new/removed/lastmod deltas.
• sitemap-image-extractor — image:image namespace extraction with BFS.
• rss-feed-delta-monitor — RSS/Atom feed item deltas.
• xml-feed-parser — generic RSS/Atom detection and parsing.

Support

Open an issue on the Actor's Apify Store page (Report an issue link on the right rail) or contact the developer through the Store profile.

License

MIT — see Apify Store page for the canonical text.