Pricing

$1.00 / 1,000 results

Try for free

Go to Apify Store

Sitemap URL Extractor

Try for free

Extract every page URL from any website sitemaps (robots.txt, sitemap indexes, gzipped and text sitemaps) with lastmod, changefreq and priority. Fast, no browser, pay per result.

Pricing

$1.00 / 1,000 results

Rating

0.0

(0)

Developer

Cody Coulter

Actor stats

Bookmarked

Total users

Monthly active users

6 days ago

Last modified

Use cases

SEO audits, content inventories, migration checklists, feeding URL lists to crawlers or RAG/LLM pipelines, monitoring newly published pages via lastmod.

Input

Field	Type	Default	Description
`startUrls`	string[]	—	Domains, homepages, or sitemap URLs (required)
`maxUrls`	integer	1000	Total URL cap across all inputs
`maxSitemaps`	integer	200	Max sitemap files followed per domain
`urlFilter`	string	—	JavaScript regex; only matching URLs are output
`includeMetadata`	boolean	true	Include lastmod/changefreq/priority/sitemap

{ "startUrls": ["docs.apify.com"], "maxUrls": 500, "urlFilter": "/platform/" }

Output (one row per URL)

{ "url": "https://docs.apify.com/api", "lastmod": null, "changefreq": "weekly", "priority": 0.5, "sitemap": "https://docs.apify.com/sitemap_base.xml" }

Export as JSON, CSV, Excel, or via API.

Pricing

Pay per result: you pay a small fee per URL in the dataset. Set maxUrls to cap your spend.

Notes

Only reads publicly published sitemaps and robots.txt — files websites provide specifically for automated discovery. No login, no page content scraping.

Local development

npm install
npm test          # end-to-end test against a local fixture site (index + .gz + dedup)
apify run         # uses storage/key_value_stores/default/INPUT.json

Sitemap Detector

coder_zoro/sitemap-detector

Find sitemap URLs fast with our free Sitemap Finder tool. Instantly detect sitemaps from any website for SEO audits, indexing checks, and crawl planning. Improve visibility, site structure insights, and search engine performance in just seconds

Zoro

182

5.0

(3)

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

649

5.0

(3)

Sitemap Extractor

cerebral_aluminum/sitemap-extractor

Extract all URLs from website sitemaps. Pages, images, PDFs. Handles sitemap indexes and WordPress.

Benny

Wayback Machine URL Extractor - Archived URLs

logiover/wayback-machine-url-extractor

Extract every archived URL of any domain from the Internet Archive's Wayback Machine (CDX API). Recover lost or old pages, build redirect maps and run OSINT, with date and status filters. No API key, export to CSV or JSON.

Logiover

Sitemap URL Extractor

sootesting/sitemap-url-extractor

Extract URLs from XML sitemaps in bulk. Supports sitemap indexes, auto-discovers nested sitemaps, and outputs structured JSON. Pay-per-event: per 1,000 URLs extracted.

soot

Sitemap URL Extractor — recursive, filters, .gz

x402opklaar/sitemap-url-extractor

Extract every URL from any sitemap: recursive sitemap-index expansion, gzipped sitemaps, robots.txt auto-discovery, include/exclude filters, lastmod output. Feed the URL list to any crawler or audit. Pay per sitemap file processed.

Opklaar

Sitemap URL Extractor - List All URLs in a Sitemap

dltik/sitemap-url-extractor

Extract every URL from any XML sitemap, with lastmod, changefreq and priority. Resolves sitemap indexes recursively. Pass a sitemap.xml or just a site root to auto-discover its sitemaps. Pure HTTP, no browser — fast and cheap.

Walid

Sitemap URL Harvester

mahogany_songbird/sitemap-url-harvester

Collect URLs from XML sitemaps for SEO and crawling.

Britton Furness

Sitemap Extractor

smorgi_apps/sitemap-extractor

Smorgi Apps

Sitemap URL Extractor

gifted_wagon/sitemap-url-inventory

Traverse public sitemap indexes into a deduplicated URL inventory with source sitemap, lastmod, changefreq, priority, media counts, and partial-failure evidence.