Pricing

Pay per usage

Robots.txt Analyzer

Analyze robots.txt for any list of domains — crawl rules per user-agent, declared sitemaps, crawl-delay, and indexing red flags (e.g. 'Disallow: /' blocking the whole site). One clean report per domain. Pay-per-event: $0.01 per batch of up to 200 domains.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

soot

Actor stats

Bookmarked

Total users

Monthly active users

6 hours ago

Last modified

Categories

SEO tools

Developer tools

Automation

This Actor‘s pricing details are outlined in the table below. The unit prices you pay depend on your Apify subscription plan, with higher tiers offering better rates. Visit our pricing page to compare plans and see how much you can save.

Pricing	Free No discount	Starter Bronze discount	Scale Silver discount	Business Gold discount
Compute units (CU)	$0.20 / CU	$0.20 / CU	$0.16 / CU	$0.13 / CU
Actor RAM	16 GB	64 GB	256 GB	512 GB
Max concurrent runs	25	32	128	256
Rented Actors		Deducted from prepaid usage	Deducted from prepaid usage	Deducted from prepaid usage
Residential proxies	$8.00 / GB	$8.00 / GB	$7.50 / GB	$7.00 / GB
Datacenter proxies	5 IPs included	30 IPs included then $1.00 / IP	200 IPs included then $0.80 / IP	500 IPs included then $0.60 / IP
SERPs proxy	$2.50 / 1,000 SERPs	$2.50 / 1,000 SERPs	$2.00 / 1,000 SERPs	$1.70 / 1,000 SERPs

This Actor's pricing details are outlined in the table below. The unit prices you pay depend on your Apify subscription plan, with higher tiers offering better rates. Visit our pricing page to compare plans and see how much you can save.

Costs
Platform usage Platform usage costs are tied to the Actor's performance in terms of the underlying platform resources it consumes. Learn more	Variable costs cheaper for higher plans.

Robots Txt Analyzer

zerobreak/robots-txt-analyzer

Robots txt analyzer that fetches and parses crawl rules from any website in bulk, so SEO teams and developers can audit blocked paths, user agents, and sitemap locations across hundreds of domains without manual work.

ZeroBreak

robots.txt Analyzer

eliai/robots-txt-analyzer

Anthony Snider

Robots.txt Analyzer

mahogany_songbird/robots-txt-analyzer

Read robots.txt disallow rules and sitemap declarations.

Britton Furness

Robots Sitemap Analyzer - SEO Crawl Rules

benthepythondev/robots-sitemap-analyzer

Analyze robots.txt files and discover sitemap URLs, user-agent groups, allow rules, disallow rules and crawl-delay directives.

ben

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Stas Persiianenko

Domain Age Checker

automation-lab/domain-age-checker

Check domain age and registration dates in bulk using RDAP — the modern WHOIS replacement. Returns registration date, expiration date, registrar, nameservers, and age in days/years. Ideal for SEO analysis, fraud detection, and domain research.

Stas Persiianenko

Sitemap URL Extractor

automation-lab/sitemap-url-extractor

This actor parses XML sitemaps and extracts all URLs with their metadata. It handles both regular sitemaps and sitemap indexes (recursively follows child sitemaps up to 3 levels deep). For each URL, it captures the last modified date, change frequency, priority, and whether the entry...

Stas Persiianenko

Redirect Chain Analyzer

automation-lab/redirect-chain-analyzer

This actor follows HTTP redirects step-by-step and reports the full chain for any list of URLs. It also tests HTTP-to-HTTPS and www/non-www variants to verify canonical redirects are properly configured. Use it for SEO migration audits, redirect debugging, or monitoring redirect health.

Stas Persiianenko

Robots.txt Checker & Parser - Crawl Rules API

pink_comic/robots-txt-validator

Check, parse, and validate robots.txt files in bulk. Extract crawl rules, sitemaps, crawl-delay, blocked paths, and per-user-agent allow/disallow results for SEO audits and crawler compliance.

Ava Torres

Sitemap Diff Tool

automation-lab/sitemap-diff-tool

Compare two XML sitemaps and find added, removed, or changed URLs. Detects lastmod, priority, and changefreq changes. Supports sitemap index files. Export results as JSON, CSV, or Excel.

Stas Persiianenko