Indexability Audit avatar

Indexability Audit

Pricing

$150.00 / 1,000 completed audits

Go to Apify Store
Indexability Audit

Indexability Audit

Can search engines index this page, and if not, why? Checks the meta robots tag, X-Robots-Tag header, canonical link, and robots.txt in one call and returns a plain verdict with the exact directive responsible. Charged only on completed audits.

Pricing

$150.00 / 1,000 completed audits

Rating

0.0

(0)

Developer

Chris Arsenault

Chris Arsenault

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

2 days ago

Last modified

Share

"Why isn't this page showing up in Google?" — answered in one call. This Actor checks every reason a page can be excluded from search and tells you which one applies: the meta robots tag, the X-Robots-Tag HTTP header, the canonical link pointing elsewhere, and a robots.txt disallow — each a common, easy-to-miss cause of pages silently vanishing from search.

What you get

{
"url": "https://example.com/page/",
"verdict": "NOT indexable",
"blockers": ["noindex directive present (X-Robots-Tag header); this page is explicitly excluded from search"],
"warnings": ["canonical points elsewhere (https://example.com/other/); search may index that URL instead"],
"signals": {
"x_robots_tag": "noindex",
"meta_robots": null,
"canonical": "https://example.com/other/",
"robots_txt_disallow": null
}
}

The verdict is plain (indexable / not indexable); the blockers name the exact directive so the fix is obvious.

Why it is more useful than a generic checker

It separates the two things people conflate: a noindex (which removes a page from the index) versus a robots.txt disallow (which blocks crawling but does not deindex an already-known URL). It also catches header-level X-Robots-Tag directives that never appear in the HTML and are the hardest to spot by eye.

Use cases

  • SEO triage: the first check when a page won't rank or disappeared from search.
  • Publishing safeguards: run after a deploy to catch a staging noindex that shipped to production (a classic, costly mistake).
  • Agents managing sites: verify a new page is indexable before waiting weeks to discover it wasn't.

Honesty notes

  • Reads the page, its response headers, and robots.txt; nothing is modified.
  • Reflects the directives as sent to a normal client; it does not guarantee Google's eventual behavior, which also depends on content quality and crawl budget.
  • Unreachable pages report cleanly and you are not charged; billing is per completed audit.

Built by 1450 Enterprises, alongside the AI Access Audit and Structured Data Audit.