Pricing

from $0.73 / 1,000 url returneds

Try for free

Go to Apify Store

Sitemap Scraper - Find the Sitemap and Watch for New URLs

Try for free

For SEO audits, crawl seeding and keeping a RAG index fresh: give it a domain and get one row per URL with lastmod, change frequency and priority. Asking for /sitemap.xml fails on real sites - stripe.com answers 404 - so it reads robots.txt, follows nested indexes and unpacks gzip. No API key.

Pricing

from $0.73 / 1,000 url returneds

Rating

0.0

(0)

Developer

NeverEmpty

Actor stats

Bookmarked

Total users

Monthly active users

7 days ago

Last modified

Categories

SEO tools

Developer tools

Automation

You can access the Sitemap Scraper - Find the Sitemap and Watch for New URLs programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = { "sites": [
9        "stripe.com",
10        "vercel.com",
11    ] }
12
13# Run the Actor and wait for it to finish
14run = client.actor("neverempty/sitemap-finder-monitor").call(run_input=run_input)
15
16# Fetch and print Actor results from the run's dataset (if there are any)
17print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
18for item in client.dataset(run.default_dataset_id).iterate_items():
19    print(item)
20
21# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Sitemap Finder & Scraper - Get Every URL, Watch for New Ones API in Python

The Apify API client for Python is the official library that allows you to use Sitemap Scraper - Find the Sitemap and Watch for New URLs API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Sitemap Scraper - Find the Sitemap and Watch for New URLs API in JavaScript

Sitemap Scraper - Find the Sitemap and Watch for New URLs API through CLI

Sitemap Scraper - Find the Sitemap and Watch for New URLs OpenAPI definition

Sitemap Scraper - Find the Sitemap and Watch for New URLs API

YouTube Most Replayed Scraper - Heatmap Peaks & Timestamps

neverempty/youtube-most-replayed-scraper

Find the most replayed moments of any YouTube video: the top peaks of YouTube's own Most replayed heatmap as timestamped links, the chapter each peak falls in and the caption line spoken there, plus the raw heatmap segments. Videos without a heatmap are listed free.

NeverEmpty

Sitemap URL Extractor

datalantern/sitemap-extractor

Extract every URL from any website's XML sitemaps, including sitemap indexes, gzip sitemaps, and robots.txt discovery. Returns lastmod, changefreq, and priority.

Data Lantern

Sitemap URL Extractor

lost.fany/sitemap-url-extractor

Extract every URL from a website's sitemap.xml, sitemap index and robots.txt in seconds — with lastmod, changefreq, priority and hreflang. No crawling, no browser: fast, cheap, reliable.

Natalia Ivanova

Sitemap URL Extractor

palgenius/sitemap-url-extractor

Extract every URL from a website's sitemaps: indexes, .gz, plain-text and RSS/Atom, with lastmod filtering.

Ali Alsaidi

Sitemap URL Extractor — Nested Indexes, Gzip & lastmod

relevate/sitemap-urls

Turn any website into a clean list of its URLs. Finds the sitemap from robots.txt, follows nested sitemap indexes, unpacks .xml.gz, reads RSS/Atom feeds, and can return only pages changed since a date — the cheap way to re-crawl just what is new.

Relevate

Sitemap URL Extractor — robots.txt, Sitemap Index, gzip

chorelet/sitemap-url-extractor

Every URL of a website from its sitemaps: robots.txt discovery, sitemap indexes, gzip, lastmod, changefreq, priority, hreflang alternates, image/video/news extensions. Include/exclude patterns and date filter, CSV/JSON export and API.

Chorelet

Sitemap Indexability & Change Monitor

produkdigitalali/sitemap-indexability-change-monitor

Audit XML sitemap URLs for HTTP status, robots directives, canonical targets, indexability, and added, removed, or changed pages across repeat runs.

ProdukDigitalAli

Sitemap Delta Monitor | New & Changed URLs

sbio-apps/sitemap-delta-monitor

Compare XML sitemap snapshots for new URLs, metadata edits and entries absent from the latest document. Keep persistent history, return the current inventory and deduplicatable change IDs, and distinguish URL sets from sitemap indexes.

Austin Aryain

Sitemap URLs: Exact File, No Page Crawl

jmspwr/sitemap-url-extractor

Extract the exact sitemap file you name, or discover a domain's sitemaps. Deduplicated URL, lastmod and source rows with explicit partial/error reports. No page crawl or proxy; empty runs are free.

James Power

Sitemap URL Exporter — Nested Sitemaps & Robots Discovery

gertner-data/sitemap-url-exporter

Export up to 3,000 deduplicated URLs per website from robots.txt and nested XML sitemaps, with lastmod and source sitemap. Feed crawlers, migrations and ETL from one JSON report. Launch price: $1/1,000 website reports; all exported URLs included.