Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps
Pricing
from $0.10 / 1,000 url extracteds
Go to Apify Store

Sitemap URL Extractor: robots.txt, Index & Gzip Sitemaps
Get every URL from a website's sitemaps. Auto-discovers sitemaps via robots.txt and common paths, follows sitemap indexes, reads .xml, .xml.gz, .txt, RSS and Atom, and de-duplicates. Filter by pattern or lastmod. Pay only per unique URL delivered.
Pricing
from $0.10 / 1,000 url extracteds
Rating
0.0
(0)
Developer
Mogul Tools
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
5 days ago
Last modified
Categories
Share