Sitemap & Indexability Auditor: Noindex, Canonical, Hreflang

Find out which URLs in your sitemaps Google cannot index, and why — plus the on-page basics (titles, meta descriptions, H1) for every URL, at a fraction of the price of a full crawler. For every sitemap URL the Actor checks HTTP status and redirects, noindex in the HTML and in the X-Robots-Tag header, the canonical tag (HTML and Link header), robots.txt rules for Googlebot, hreflang and lastmod. You get a clear verdict with reasons per URL, a summary per page template, and alerts when something becomes non-indexable between scheduled runs.

Built for SEO and WordPress/WooCommerce agencies, site migrations and in-house teams that keep sitemaps clean.

What you get per URL

Field	Meaning
indexable	true, false, or null when the URL could not be reached
blockers	Why it is not indexable: http-3xx, http-4xx, http-5xx, redirect-loop, too-many-redirects, blocked-by-robots, noindex-meta, noindex-header, canonical-other, non-html, fetch-failed
warnings	Worth fixing, not blocking: canonical-missing, canonical-conflict, canonical-not-absolute-match, hreflang-invalid-code, hreflang-missing-self, hreflang-missing-x-default, hreflang-not-reciprocal, hreflang-target-not-indexable, lastmod-in-future, lastmod-older-than-1y, slow-response, body-truncated, robots-unavailable, title-missing, title-too-long (>60 chars), title-duplicate, meta-description-missing, meta-description-too-long (>160 chars), meta-description-duplicate, h1-missing, h1-multiple
Details	HTTP status, redirect chain, final URL, canonical and its class (self, other, missing, conflict), meta robots, X-Robots-Tag, hreflang list, title, meta description, H1s, response time
diff, regression	With a Monitor ID: what changed since the last run

The key-value store also gets SUMMARY (per site and template: indexable rate, counts per blocker and warning) and ISSUES (non-indexable URLs and regressions, ready for alerts).

Quick start

1. Enter your domain (e.g. example.com). The Actor finds the sitemaps in robots.txt or at the usual WordPress / Yoast / Rank Math locations.
2. Run it. Leave Samples per template at 0 to audit everything, or set e.g. 10 to audit 10 URLs per template on large sites.
3. For monitoring, set a Monitor ID and create an Apify Schedule (weekly works well). Connect the Not indexable only view or the ISSUES record to Slack, email, n8n or Make.

Good to know

• Polite by default: at most 4 requests per second per site, robots.txt Crawl-delay honoured, honest User-Agent IndexabilityAuditor/0.1 (Apify Actor; +https://apify.com). URLs that robots.txt blocks for Googlebot are reported without being fetched.
• Firewalls: some sites block automated requests (HTTP 403). Allow-list the User-Agent above to audit them; those URLs are reported with a clear message and are not charged.
• robots.txt rules follow RFC 9309 and Google's documented behaviour (most specific group, longest match, * and $; 4xx = allow all; 5xx or unreachable = disallow all).
• Duplicates are checked among indexable pages of the same site in the same run.
• No JavaScript rendering: tags injected by JavaScript are not seen (most SEO plugins output them server-side).
• Large sitemaps: up to 50 sitemap files and 50,000 URLs are read per run; files you name explicitly are read first.

Pricing (pay per event)

Event	When
url-audited	One URL with a verdict delivered

Never charged: URLs that could not be reached (timeouts, network errors), invalid input, and anything beyond your max-charge limit — the run audits only as many URLs as your limit covers. (Prices: see the Pricing tab.)

Related

• Core Web Vitals Monitor: real-user CrUX data, Lighthouse lab audits, trends and regression alerts per page template.

Support

Open an issue on the Actor's Issues tab with the run ID.