Pricing

from $0.75 / 1,000 urls

Try for free

Go to Apify Store

Sitemap Scraper: Every URL on Any Site

Try for free

Get every URL a website publishes, from a bare domain: a website pages list from its sitemap.xml. Finds the sitemap through robots.txt, follows nested sitemap index files to any depth, unpacks .xml.gz, returns each URL with lastmod and priority. Filter by path regex or by date.

Pricing

from $0.75 / 1,000 urls

Rating

0.0

(0)

Developer

Daniel Meshulam

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Categories

SEO tools

Developer tools

Automation

Websites

startUrls

Required

Sites to pull the sitemap from, one per line. A bare domain like apify.com is enough, since the sitemap is discovered from robots.txt or the usual paths. You can also paste a sitemap URL directly.

Type:array

Max URLs per site

maxUrlsPerSite

Optional

Stop after this many URLs per site. Nested sitemap index files are followed automatically until this cap is hit. The form is prefilled with 50 so a first run costs cents; the cap when this field is left out entirely is 5000.

Type:integer

Minimum:1

Maximum:200000

Default:5000

Only paths matching

includePattern

Optional

Regular expression tested against the URL path. Example: ^/blog/ keeps only blog posts.

Type:string

Skip paths matching

excludePattern

Optional

Regular expression tested against the URL path. Example: /(tag|author|page)/ drops paginated and taxonomy pages.

Type:string

Changed since

changedSince

Optional

Only URLs whose is on or after this date (YYYY-MM-DD). URLs with no lastmod are dropped. Use it to pull just what changed since your last crawl.

Type:string

Proxy

proxyConfiguration

Optional

Proxy settings.

Type:object

Default:

{
  "useApifyProxy": true
}

Maximum results (whole run)

maxItems

Optional

A hard ceiling on rows for the entire run, across every target. The per-target limits above cap each one separately, so fifty targets at twenty each is still a thousand rows; this caps the total. The run stops cleanly when it is reached and nothing beyond that point is fetched or charged. Leave empty for no ceiling.

Type:integer

Minimum:0

Sitemap Generator - Crawl Website & Create XML Sitemap

scrappy_garden/sitemap-generator

Generate an XML sitemap for any website. Crawls internal pages from start URLs (with depth + page limits), deduplicates URLs, and stores a ready-to-submit sitemap.xml plus a structured dataset and summary for SEO audits.

Bikram Adhikari

Website Sitemap Extractor

glassventures/website-sitemap-extractor

Extract all URLs from any website's sitemap. Auto-discovers sitemaps from robots.txt, supports sitemap index files and .gz compression. Filter by URL pattern, date range.

Glass Ventures

YouTube Channel Stats: Subscribers, Views, No API Key

glitchbound/youtube-channel-stats

YouTube channel statistics without an API key: subscriber count, total views, video count, age, country and contact links for any channel. Vet YouTube influencers by last upload date, so a dormant channel never makes your list. Also an MCP tool for AI agents. No quota.

Daniel Meshulam

Sitemap Generator

datawinder/sitemap-generator

Automatically crawl a website and generate an SEO-ready sitemap in XML, HTML, or TXT format. Supports crawl depth limits, URL include/exclude patterns, and optional merging with an existing sitemap.xml. Ideal for SEO audits, site migrations, and automation.

DatawinderLabs

Sitemap Change Detector

tri_angle/sitemap-change-detector

Identify and monitor sitemaps for specified websites. Retrieve only the new, updated, or removed URLs since the last crawl.

Tri⟁angle

Sitemap URL Extractor & XML Sitemap Checker

somepineapple/sitemap-extractor-checker-tool

Get every URL from a website's XML sitemaps and spot problems with them. Finds the sitemap from robots.txt or /sitemap.xml, follows sitemap indexes and .gz files, returns lastmod, changefreq and priority, and flags duplicates, bad dates and broken sitemaps.

Ollie Wood

Sitemap Finder · URLs from Any XML

corent1robert/sitemap-detector

Find and crawl XML sitemaps from any website. Follows sitemap indexes, handles gzip, and exports every page URL with source file and lastmod into a clean dataset. No config needed.

Corentin Robert

Sitemap URL Extractor — recursive, filters, .gz

x402opklaar/sitemap-url-extractor

Extract every URL from any sitemap: recursive sitemap-index expansion, gzipped sitemaps, robots.txt auto-discovery, include/exclude filters, lastmod output. Feed the URL list to any crawler or audit. Pay per sitemap file processed.

Opklaar

Sitemap URL Harvester

mahogany_songbird/sitemap-url-harvester

Collect URLs from XML sitemaps for SEO and crawling.

Britton Furness

App Store & Google Play Scraper: Apps, Ratings & Reviews

glitchbound/app-store-scraper

Scrape the Apple App Store and Google Play: search rankings, ratings, review counts, price, version, developer, installs, and full user reviews. Built on official endpoints for reliability.

Daniel Meshulam