Sitemap Scraper: Every URL on Any Site avatar

Sitemap Scraper: Every URL on Any Site

Pricing

from $0.75 / 1,000 urls

Go to Apify Store
Sitemap Scraper: Every URL on Any Site

Sitemap Scraper: Every URL on Any Site

Get every URL a website publishes, from a bare domain: a website pages list from its sitemap.xml. Finds the sitemap through robots.txt, follows nested sitemap index files to any depth, unpacks .xml.gz, returns each URL with lastmod and priority. Filter by path regex or by date.

Pricing

from $0.75 / 1,000 urls

Rating

0.0

(0)

Developer

Daniel Meshulam

Daniel Meshulam

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

a day ago

Last modified

Share