Pricing

from $0.40 / 1,000 url extracteds

Try for free

Go to Apify Store

Sitemap URL Extractor - All Website URLs, New & Removed Pages

Try for free

Get every URL of any website from its sitemap.xml: nested sitemap indexes, .xml.gz, text sitemaps, robots.txt discovery. Export to CSV, JSON or Excel. Track new and removed pages between runs. Optional 404 status check. Streams millions of URLs. Pay per URL, no proxy needed.

Pricing

from $0.40 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Luca Pietrini

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Sitemap URL Extractor: All Website URLs to CSV/JSON API through CLI

The Apify CLI is the official tool that allows you to use Sitemap URL Extractor - All Website URLs, New & Removed Pages locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

Sitemap URL Extractor - All Website URLs, New & Removed Pages API in Python

Sitemap URL Extractor - All Website URLs, New & Removed Pages API in JavaScript

Sitemap URL Extractor - All Website URLs, New & Removed Pages OpenAPI definition

Sitemap URL Extractor - All Website URLs, New & Removed Pages API

Sitemap Scraper

clintsa/sitemap-scraper

Extract all URLs from any website's sitemap.xml with robots.txt discovery, gzip support, full metadata, and automatic retries

Andy Besos

Sitemap URL Extractor

automation-lab/sitemap-url-extractor

This actor parses XML sitemaps and extracts all URLs with their metadata. It handles both regular sitemaps and sitemap indexes (recursively follows child sitemaps up to 3 levels deep). For each URL, it captures the last modified date, change frequency, priority, and whether the entry...

Automation Lab

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Automation Lab

Sitemap to URL Crawler — Extract Sitemap.xml URLs

logiover/sitemap-to-url-crawler

Extract all URLs from any sitemap.xml recursively. Export sitemap URLs to CSV/JSON for RAG pipelines, SEO audits, and LLM training datasets.

Logiover

Sitemap Extractor & URL Change Monitor

automa-flow/sitemap-extractor-monitor

Extract every URL from XML, gzip, text and nested sitemaps, with robots.txt discovery. Compare repeat checks for added URLs, publisher lastmod updates and confirmed removals. An incomplete sitemap tree never produces false removals. Export for SEO audits and incremental crawlers.

Vadim Bezrukov

Sitemap Change Orchestrator

tri_angle/sitemap-change-orchestrator

Monitor website sitemaps for new, updated, or removed URLs. Integration with the Website Content Crawler (WCC) allows feeding only relevant URLs. This ensures your web crawls are efficient, targeted, and resource-optimized, keeping your datasets fresh for any application.

Tri⟁angle

Social Media Profile Finder

automation-lab/social-media-profile-finder

Find social media profiles linked from any website — Facebook, Twitter/X, LinkedIn, Instagram, YouTube, TikTok, and more.

Automation Lab

388

4.0

(2)

Sitemap URL Extractor — Every URL in a sitemap.xml as JSON

eliai/sitemap-url-extractor

Get every URL out of a sitemap.xml, a sitemap index (followed recursively), or a bare domain via robots.txt. Returns loc, lastmod, changefreq and priority per URL. Priced per FILE, not per URL: $0.005 per sitemap plus $0.015 per 1,000 URLs. Failures never charged.

Broke to Built

Bulk URL Status Checker

automation-lab/bulk-url-status-checker

Bulk check URLs for status codes, redirects, broken links, response times, canonical tags, robots meta, headers, and final destinations.

Automation Lab

Bulk URL Status Checker – Broken Link & Redirect Audit

logiover/bulk-url-status-checker

Bulk HTTP status code checker and broken link checker. Trace redirect chains, find 404s, export to CSV/JSON. No browser, no login.

Logiover

552

Bulk Website Contact Extractor & Email Finder

harvestlab/contact-extractor

Bulk-enrich company domains with public emails, phones, social profiles, contact pages, addresses, and tech-stack signals. Export CRM-ready lead records with optional email verification and role labels.