Sitemap URL Extractor for AI Agents
Pricing
$0.10 / 1,000 sitemap urls
Sitemap URL Extractor for AI Agents
Discover and expand sitemaps into crawl URLs. Price: $0.0001 per emitted URL.
Pricing
$0.10 / 1,000 sitemap urls
Rating
0.0
(0)
Developer
Gavin Coates
Maintained by CommunityActor stats
0
Bookmarked
1
Total users
0
Monthly active users
2 hours ago
Last modified
Categories
Share
Deterministic sitemap discovery and URL extraction for AI agents, crawlers, RAG pipelines, SEO tooling, and automated workflows.
Actor: industrial_platform/sitemap-intelligence
Price
$0.0001 per unique emitted sitemap URL using the sitemap-url pay-per-event billing event.
Failed sitemap fetches do not produce URL billing events.
Examples:
- 1 URL: $0.0001
- 1,000 URLs: $0.10
- 10,000 URLs: $1.00
- 100,000 URLs: $10.00
Discovery behavior
For a domain or website URL, the Actor:
- checks
robots.txtfor Sitemap directives; - tries
/sitemap.xmlas a fallback; - follows sitemap indexes recursively;
- supports XML,
.xml.gz, plain-text URL lists, RSS, and Atom; - emits one deduplicated row per discovered URL.
Input
A start may be a domain, website URL, or direct sitemap URL.
Output
Each URL row can include:
- URL
lastmodchangefreqpriority- source sitemap
- discovery timestamp
When to use it
Use this Actor for sitemap extractor, sitemap URL enumeration, crawl planning, website inventory, RAG ingestion planning, sitemap-index expansion, and SEO automation.
Limits
Default limits:
- 20 starts
- 100 sitemap documents
- 50,000 unique URL rows
Hard maximum:
- 500 sitemap documents
- 100,000 URL rows
Only public HTTP/HTTPS resources are allowed; private/local network targets are blocked.
MCP
The required MCP/API input is start_urls.
