Sitemap URL Extractor & Sitemap Scraper - All Website URLs

Sitemap Extractor

apify/sitemap-extractor

This Apify Actor extracts all URLs from a website's sitemaps and checks their status codes via lightweight HTTP requests. It provides a clean list of valid links, acting as an ideal pre-processor to ensure your larger crawling projects target only active URLs.

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

Generate an XML sitemap for any website. Crawls internal pages from start URLs (with depth + page limits), deduplicates URLs, and stores a ready-to-submit sitemap.xml plus a structured dataset and summary for SEO audits.

Website Metadata Extractor (meta tags, sitemap, robots) 🔎

powerful_bachelor/website-metadata-extractor

🔍 Website Metadata Extractor 🌐 Extract essential website data: meta tags, robots.txt, and sitemap.xml in one scan. 📊 Analyze SEO elements, crawler directives, and site structure. ✅ Perfect for SEO audits, 🔎 competitor research, and 🚀 understanding how search engines view your website.

Sitemap URL Extractor & 404 Checker — XML Sitemap Scraper

factpipe/sitemap-url-extractor

Extract every URL from any website's XML sitemaps: robots.txt discovery, nested sitemap indexes, lastmod and priority. Optional HTTP status check finds 404s and redirects. $0.30 per 1,000 URLs.

Sitemap URL Extractor

gifted_wagon/sitemap-url-inventory

Traverse public sitemap indexes into a deduplicated URL inventory with source sitemap, lastmod, changefreq, priority, media counts, and partial-failure evidence.

Extract every URL from any website's XML sitemaps. Auto-discovers sitemaps via robots.txt and common paths, follows nested sitemap indexes, supports .gz and text sitemaps, regex filters. Returns lastmod, changefreq, priority, hreflang alternates and image counts.

Website Sitemap URL Extractor

myagizm/website-sitemap-extractor

Extract and deduplicate page URLs with metadata from a public XML sitemap or bounded sitemap index.

Sitemap URL Extractor: All Pages of Any Website

jtpalms/sitemap-url-extractor

Get every URL of a website from its XML sitemaps. Auto-discovers sitemaps from robots.txt, follows sitemap indexes and .gz files, and returns lastmod, changefreq, priority and image counts. Filter by URL pattern or last-modified date. USD 0.50 per 1,000 URLs.

SSL Certificate Checker: Bulk Expiry & Domain Expiry Monitor

datawright-labs/ssl-certificate-checker

Check SSL/TLS certificates for thousands of domains at once: days until expiry, issuer, SANs, chain, trust and hostname errors, TLS version and cipher, plus domain registration expiry via RDAP. Flags expired, expiring and invalid certs.