Pricing

from $0.10 / 1,000 url extracteds

Try for free

Go to Apify Store

Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps

Try for free

Get every URL from a website's sitemaps. Auto-discovers sitemaps via robots.txt and common paths, follows sitemap indexes, reads .xml, .xml.gz, .txt, RSS and Atom, and de-duplicates. Filter by pattern or lastmod. Pay only per unique URL delivered.

Pricing

from $0.10 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Mogul Tools

Actor stats

Bookmarked

Total users

Monthly active users

5 days ago

Last modified

Categories

SEO tools

Developer tools

You can access the Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "startUrls": [
12        "https://www.djangoproject.com",
13        "https://nodejs.org"
14    ],
15    "proxyConfiguration": {
16        "useApifyProxy": false
17    }
18};
19
20// Run the Actor and wait for it to finish
21const run = await client.actor("blond_terrarium/sitemap-url-extractor-pro").call(input);
22
23// Fetch and print Actor results from the run's dataset (if any)
24console.log('Results from dataset');
25console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
26const { items } = await client.dataset(run.defaultDatasetId).listItems();
27items.forEach((item) => {
28    console.dir(item);
29});
30
31// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Sitemap URL Extractor & sitemap.xml Parser API API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps API in Python

Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps API through CLI

Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps OpenAPI definition

Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps API

Sitemap Extractor

apify/sitemap-extractor

This Apify Actor extracts all URLs from a website's sitemaps and checks their status codes via lightweight HTTP requests. It provides a clean list of valid links, acting as an ideal pre-processor to ensure your larger crawling projects target only active URLs.

Apify

414

3.2

(8)

Sitemap Detector

coder_zoro/sitemap-detector

Find sitemap URLs fast with our free Sitemap Finder tool. Instantly detect sitemaps from any website for SEO audits, indexing checks, and crawl planning. Improve visibility, site structure insights, and search engine performance in just seconds

Zoro

182

5.0

(3)

Sitemap Generator - Crawl Website & Create XML Sitemap

scrappy_garden/sitemap-generator

Generate an XML sitemap for any website. Crawls internal pages from start URLs (with depth + page limits), deduplicates URLs, and stores a ready-to-submit sitemap.xml plus a structured dataset and summary for SEO audits.

Bikram Adhikari

Sitemap URL Extractor & 404 Checker — XML Sitemap Scraper

factpipe/sitemap-url-extractor

Extract every URL from any website's XML sitemaps: robots.txt discovery, nested sitemap indexes, lastmod and priority. Optional HTTP status check finds 404s and redirects. $0.30 per 1,000 URLs.

Neo B

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

653

5.0

(3)

Sitemap URL Extractor

sootesting/sitemap-url-extractor

Extract URLs from XML sitemaps in bulk. Supports sitemap indexes, auto-discovers nested sitemaps, and outputs structured JSON. Pay-per-event: per 1,000 URLs extracted.

soot

Sitemap URL Extractor

gifted_wagon/sitemap-url-inventory

Traverse public sitemap indexes into a deduplicated URL inventory with source sitemap, lastmod, changefreq, priority, media counts, and partial-failure evidence.

Michael Olmos

Sitemap URL Extractor - XML Sitemap Scraper

benthepythondev/sitemap-url-extractor

Extract URLs from XML sitemaps and sitemap indexes. Get URL, lastmod, changefreq, priority and source sitemap.

Ben

Small Sitemap Inventory

sapph1re/small-sitemap-inventory

Build a bounded same-host URL inventory from small public HTTPS sitemaps with explicit completeness and source receipts.

Roman V

Sitemap URL Extractor — Robots.txt & Sitemap.xml Parser

tarnlight/sitemap-url-extractor

Extracts the URLs a website lists in its XML sitemaps (sitemap indexes, gzip and text sitemaps, robots.txt discovery), up to your Max URLs cap, with filters and a clear per-site report.