Sitemap Scraper - Find the Sitemap and Watch for New URLs avatar

Sitemap Scraper - Find the Sitemap and Watch for New URLs

Pricing

from $0.73 / 1,000 url returneds

Go to Apify Store
Sitemap Scraper - Find the Sitemap and Watch for New URLs

Sitemap Scraper - Find the Sitemap and Watch for New URLs

Give it a domain. It reads robots.txt to find where that site really keeps its sitemap, follows nested indexes, unpacks gzip, and returns one row per URL with lastmod, change frequency and priority. Monitoring mode returns only the URLs that are new since the last run.

Pricing

from $0.73 / 1,000 url returneds

Rating

0.0

(0)

Developer

NeverEmpty

NeverEmpty

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

2 days ago

Last modified

Share