Pricing

from $0.30 / 1,000 urls

Try for free

Go to Apify Store

Sitemap URL Extractor: Every Page, Lastmod & New URLs

Try for free

Extract every URL from any website's XML sitemaps. Auto-discovers sitemaps via robots.txt, follows sitemap indexes, reads .gz, text and news/image/hreflang sitemaps. Returns lastmod, priority and images, filters by pattern or date, and flags new URLs since the last run.

Pricing

from $0.30 / 1,000 urls

Rating

0.0

(0)

Developer

Lantern Lane Data

Actor stats

Bookmarked

Total users

Monthly active users

17 days ago

Last modified

Sitemap URL Extractor: All Website URLs from XML Sitemaps API through CLI

The Apify CLI is the official tool that allows you to use Sitemap URL Extractor: Every Page, Lastmod & New URLs locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Sitemap URL Extractor: Every Page, Lastmod & New URLs API in Python

Sitemap URL Extractor: Every Page, Lastmod & New URLs API in JavaScript

Sitemap URL Extractor: Every Page, Lastmod & New URLs OpenAPI definition

Sitemap URL Extractor: Every Page, Lastmod & New URLs API

Sitemap Extractor

apify/sitemap-extractor

This Apify Actor extracts all URLs from a website's sitemaps and checks their status codes via lightweight HTTP requests. It provides a clean list of valid links, acting as an ideal pre-processor to ensure your larger crawling projects target only active URLs.

Apify

403

3.2

(8)

Website Sitemap Extractor

glassventures/website-sitemap-extractor

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

Glass Ventures

XML Sitemap Finder & Extractor API

andok/find-sitemap-from-url

Find and extract all XML sitemaps for any domain. Automatically parses robots.txt, scans HTML tags, and recursively follows indexes. Perfect for SEO & web scraping.

Andok

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Automation Lab

Sitemap URL Harvester

mahogany_songbird/sitemap-url-harvester

Collect URLs from XML sitemaps for SEO and crawling.

Britton Furness

Sitemap Change Monitor: New & Removed Page Alerts

scrapemint/sitemap-change-monitor

Watch any website's sitemap and get one row per change: new pages, removed pages, optional lastmod updates. See competitors' new products, content clusters and landing pages the day they ship. Pay only for changes; quiet sites cost nothing. Keyless, no proxy.

Ken M

5.0

(1)

Sitemap URL Extractor – All URLs from sitemap.xml & robots.txt

openharvest/sitemap-url-extractor

Extract page URLs from a website's sitemaps. Finds sitemaps via robots.txt, common paths and homepage links, then reads XML and .xml.gz sitemaps, sitemap indexes, text sitemaps, RSS and Atom feeds. Returns lastmod, changefreq, priority, images, videos and hreflang.

Enes Bulanık

Website Metadata Extractor (meta tags, sitemap, robots) 🔎

powerful_bachelor/website-metadata-extractor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

Powerful Bachelor

App Store Top Charts & Search Scraper - All Countries

olsols4/app-store-search-top-charts

Top free, paid and grossing App Store charts for all 155 countries and every category, plus keyword search results and full app metadata. Track your app's rank across countries in one run. No proxy, no login.

Olly

Sitemap URL Extractor & Sitemap Scraper: All Website URLs

nebulacolors/sitemap-urls

Sitemap extractor that lists all website URLs from its sitemaps: robots.txt Sitemap lines or /sitemap.xml, nested sitemap indexes, gzip and text sitemaps, with lastmod, changefreq and priority. Filter by pattern or date, or crawl links when there's no sitemap. $0.0003 per URL.