Sitemap URL Extractor: All Pages from Any Website

Get every URL a website publishes in its XML sitemaps, with no login, no browser and no code. Export to JSON, CSV or Excel, or use the API.

Get every URL a website publishes in its XML sitemaps: auto-discovers sitemaps from robots.txt, follows sitemap indexes, reads .gz files, and returns lastmod, changefreq, priority and image counts. Filter by URL pattern or 'modified after' date. Bulk: thousands of sites per run. Perfect input for crawlers, SEO audits and content inventories.

Use cases

• Feed a crawler or the RAG/Markdown tools exactly the pages you want
• Content inventory and 'what changed since' lists for SEO
• Find competitors' new product or blog pages by lastmod date

Input

Field	Type	Notes
websites	array	Domains, homepages or direct sitemap.xml links.
discoverFromRobots	boolean
maxUrlsPerSite	integer
includePatterns	array	e.g. /blog/
excludePatterns	array	e.g. /tag/
lastmodAfter	string	ISO date, e.g. 2026-01-01. Leave empty for all.
includeImages	boolean
maxConcurrency	integer

Example input

{
 "websites": [
  "apify.com",
  "https://therealtour360.com",
  "https://www.python.org/sitemap.xml",
  "nonexistent-domain-zzq81.com"
 ],
 "maxUrlsPerSite": 300
}

Output

One dataset item per result. Main columns: site, url, lastmod, changefreq, priority, imageCount, sitemap, error. A RUN_SUMMARY record is saved to the default key-value store. See the sample output in the Output tab.

Example output

{
  "site": "https://apify.com",
  "url": "https://apify.com/",
  "lastmod": null,
  "changefreq": null,
  "priority": null,
  "sitemap": "https://apify.com/sitemap/pages.xml",
  "imageCount": 0
}

Pricing

Pay per result, with no monthly rental or subscription:

• URL extracted: $0.0002

Example: 1,000 results cost about $0.20 in fees. Apify's free plan includes $5 of monthly credit, which is enough to test this Actor.

FAQ

Do I need an account, cookies or a login for the source site? No. Sitemap URL Extractor uses public data sources only, so there is nothing to connect and no account to risk.

How much will a run cost? You pay only for results delivered (see Pricing above), plus Apify platform usage, which the free plan's monthly credit easily covers for test runs. Start with a small limit to see the exact cost per run.

Can I export to CSV, Excel or Google Sheets? Yes. Every run's dataset can be downloaded as JSON, CSV, Excel, XML or HTML, or pushed to Google Sheets, Make, Zapier, n8n, Slack or a webhook through Apify integrations.

Can I run it on a schedule or from my own code? Yes. Add an Apify Schedule (hourly, daily, weekly) or call it through the Apify API / Python & JavaScript clients. It also works with AI agents through the Apify MCP server.

What happens if an input is invalid or a source is down? The run keeps going. The bad input gets a row with an error field so you can see what failed, and you are not charged for results that were not delivered.

Is this legal to use? It collects publicly available data. You are responsible for using the output in line with the source's terms and privacy laws such as GDPR/CCPA; don't use it for spam.

Something missing or broken? Open an issue on the Issues tab. Feature requests are usually added within a few days.

Scheduling / monitoring

Create a Schedule (daily/weekly) and connect a webhook or Slack/email integration.

Fair use

Official public APIs only, no login. Do not use output for spam or harassment. Respect the data sources' rate limits and terms.

REST API

GET https://api.mediaprotools.com/v1/sitemap-extractor?items=... (header X-API-Key). Up to 3 items per call, separated by ,. Or POST {"items":[...],"options":{...}}.

Parameter	Type	Notes
items	token	Domains, homepages or direct sitemap.xml links.
discoverFromRobots	boolean	Discover sitemaps from robots.txt
maxUrlsPerSite	integer 1-10000	Max URLs per site
includePatterns	array	e.g. /blog/
excludePatterns	array	e.g. /tag/
lastmodAfter	string	ISO date, e.g. 2026-01-01. Leave empty for all.
includeImages	boolean	Include image URLs

Output fields: changefreq, error, imageCount, lastmod, priority, site, sitemap, url.