Pricing

from $2.49 / 1,000 results

Try for free

Go to Apify Store

Sitemap URL Scraper

Try for free

Sitemap Scraper SD - Sitemap Scraper is an SEO crawling tool that extracts every URL from a website's sitemaps (robots.txt, sitemap.xml, indexes, gzip, RSS) with lastmod, changefreq, priority, hreflang and news data.

Pricing

from $2.49 / 1,000 results

Rating

0.0

(0)

Developer

Neuro Scraper

Actor stats

Bookmarked

Total users

Monthly active users

15 days ago

Last modified

Categories

SEO tools

Developer tools

You can access the Sitemap URL Scraper programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "startUrls": [{ "url": "https://crawlee.dev" }],
10    "proxyConfiguration": { "useApifyProxy": True },
11}
12
13# Run the Actor and wait for it to finish
14run = client.actor("neuro-scraper/sitemap-url-scraper").call(run_input=run_input)
15
16# Fetch and print Actor results from the run's dataset (if there are any)
17print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
18for item in client.dataset(run.default_dataset_id).iterate_items():
19    print(item)
20
21# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Sitemap Scraper — All URLs from XML & Gzip Sitemaps API in Python

The Apify API client for Python is the official library that allows you to use Sitemap URL Scraper API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Sitemap URL Scraper API in JavaScript

Sitemap URL Scraper API through CLI

Sitemap URL Scraper OpenAPI definition

Sitemap URL Scraper API

Sitemap Detector

coder_zoro/sitemap-detector

Find sitemap URLs fast with our free Sitemap Finder tool. Instantly detect sitemaps from any website for SEO audits, indexing checks, and crawl planning. Improve visibility, site structure insights, and search engine performance in just seconds

Zoro

182

5.0

(3)

Facebook Reels Scraper

neuro-scraper/facebook-reels-scraper

Facebook Reels Scraper SD - Facebook Reels Scraper is a social media data tool that extracts reels of public Facebook pages with captions, plays, durations, audio, owners and MP4 links, plus optional comments and transcripts - Facebook reels extractor.

Neuro Scraper

5.0

(1)

XML Sitemap Finder & Extractor API

andok/find-sitemap-from-url

Find and extract all XML sitemaps for any domain. Automatically parses robots.txt, scans HTML tags, and recursively follows indexes. Perfect for SEO & web scraping.

Andok

Sitemap URL Harvester

mahogany_songbird/sitemap-url-harvester

Collect URLs from XML sitemaps for SEO and crawling.

Britton Furness

Bilibili Scraper

neuro-scraper/bilibili-scraper

Bilibili Scraper SD - Bilibili Scraper is a social media data tool that extracts public Bilibili (哔哩哔哩) videos from search, video details, full comment threads, creator videos and profiles, popular and zone rankings and hot searches, with views, likes, coins, danmaku and tags. No login.

Neuro Scraper

AutoTrader Cars Scraper

neuro-scraper/autotrader-cars-scraper

AutoTrader Cars Scraper SD - AutoTrader Cars Scraper is a logged-out tool that extracts car listings from AutoTrader UK, USA (VIN, KBB price), Canada and South Africa: price, price rating, mileage, specs, history, images and dealer details.

Neuro Scraper

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Automation Lab

Sitemap URL Extractor

gifted_wagon/sitemap-url-inventory

Traverse public sitemap indexes into a deduplicated URL inventory with source sitemap, lastmod, changefreq, priority, media counts, and partial-failure evidence.

Michael Olmos

Sitemap URL Extractor

sootesting/sitemap-url-extractor

Extract URLs from XML sitemaps in bulk. Supports sitemap indexes, auto-discovers nested sitemaps, and outputs structured JSON. Pay-per-event: per 1,000 URLs extracted.

soot

Sitemap URL Extractor & New Page Monitor

henri_dewitte/sitemap-url-extractor

Extract every URL from a website's sitemaps. Auto-finds sitemaps via robots.txt, opens indexes and .gz files, adds lastmod, hreflang, images and news data. Track new pages daily. $1 per 1,000 URLs.