Pricing

from $0.30 / 1,000 sitemap url extracteds

Try for free

Go to Apify Store

Sitemap URL Extractor: Get All URLs from a Website

Try for free

Sitemap URL extractor for websites you supply. Get all URLs from XML sitemaps, including sitemap indexes and .xml.gz files, with lastmod, priority, source and optional HTTP status. Robots.txt is respected; this actor does not crawl JavaScript pages.

Pricing

from $0.30 / 1,000 sitemap url extracteds

Rating

0.0

(0)

Developer

Weio, Inc.

Actor stats

Bookmarked

Total users

Monthly active users

20 days ago

Last modified

Categories

SEO tools

Developer tools

Automation

Websites

startUrls

Required

Websites or any URL on them, one per line. The actor finds each site's sitemaps itself (robots.txt Sitemap lines, then /sitemap.xml, /sitemap_index.xml and /wp-sitemap.xml).

Type:array

Max URLs per site

maxUrlsPerSite

Optional

Stop reading a site's sitemaps after this many output URLs (after filters).

Type:integer

Minimum:1

Maximum:50000

Default:200

Max items

maxItems

Optional

An additional per-site output limit for generic Apify integrations. The lower of this and Max URLs per site applies.

Type:integer

Minimum:1

Maximum:50000

Default:200

Only URLs modified on or after

lastmodAfter

Optional

Optional date (YYYY-MM-DD). Keeps URLs whose sitemap lastmod is on or after it; URLs without a lastmod are left out when this is set.

Type:string

URL must match

urlPattern

Optional

Optional regular expression (case-insensitive), e.g. /blog/ or .pdf$. A pattern that is not a valid regex is matched as plain text.

Type:string

Check each URL's HTTP status

checkStatus

Optional

Also request every output URL (HEAD, then GET if HEAD fails) and add its status. Slower: requests to one site are paced to about one per second and obey robots.txt.

Type:boolean

Default:false

Website URL Crawler & Link Extractor

maged120/get-urls-pro

Crawl any website and extract every URL with its depth, parent page and anchor text. Build a sitemap or audit internal links. Works on static and JavaScript sites.

Maged

222

XML Sitemap Finder & Extractor API

andok/find-sitemap-from-url

Find and extract all XML sitemaps for any domain. Automatically parses robots.txt, scans HTML tags, and recursively follows indexes. Perfect for SEO & web scraping.

Andok

Sitemap URL Harvester

mahogany_songbird/sitemap-url-harvester

Collect URLs from XML sitemaps for SEO and crawling.

Britton Furness

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a website link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

653

5.0

(3)

Sitemap URL Extractor - List All URLs in a Sitemap

dltik/sitemap-url-extractor

Extract every URL from any XML sitemap, with lastmod, changefreq and priority. Resolves sitemap indexes recursively. Pass a sitemap.xml or just a site root to auto-discover its sitemaps. Pure HTTP, no browser — fast and cheap.

Walid

Sitemap URL Extractor

gifted_wagon/sitemap-url-inventory

Traverse public sitemap indexes into a deduplicated URL inventory with source sitemap, lastmod, changefreq, priority, media counts, and partial-failure evidence.

Michael Olmos

Sitemap Extractor

smorgi_apps/sitemap-extractor

Smorgi Apps

Sitemap URL Extractor

sootesting/sitemap-url-extractor

Extract URLs from XML sitemaps in bulk. Supports sitemap indexes, auto-discovers nested sitemaps, and outputs structured JSON. Pay-per-event: per 1,000 URLs extracted.