Pricing

from $5.00 / 1,000 completed page preflights

Try for free

Go to Apify Store

AI Crawlability Preflight

Try for free

Batch-check 1–100 public pages for robots.txt, noindex, canonical, redirects, HTTP status, and JSON-LD with deterministic indexability/crawlability evidence.

Pricing

from $5.00 / 1,000 completed page preflights

Rating

0.0

(0)

Developer

SUTHEE KOSITWONGSAKUL

Actor stats

Bookmarked

Total users

Monthly active users

8 days ago

Last modified

Indexability and Canonical Audit Crawler

kayhermes/indexability-and-canonical-audit-crawler

Deterministic local indexability + canonical audit of explicit public HTTP(S) URLs from static HTML: robots.txt enforcement, meta/X-Robots-Tag noindex, HTTP status, redirect chains, and declared canonical analysis.

Khoa Nguyen

AI Crawlability Auditor | llms.txt robots & Schema Check

eesti/ai-crawlability-auditor

Score a site for AI crawlability: llms.txt, GPTBot/ClaudeBot/Perplexity robots rules, schema.org, sitemap, and JS-free content. $0.02/site + $0.02 start.

ando

AI Crawlability Auditor | llms.txt robots & Schema Check

andok/ai-crawlability-auditor

Score a site for AI crawlability: llms.txt, GPTBot/ClaudeBot/Perplexity robots rules, schema.org, sitemap, and JS-free content. $0.02/site + $0.02 start.

Andok

Robots & Sitemap Analyzer

ib4ngz/robots-sitemap-analyzer

Analyze robots.txt and XML sitemaps into normalized crawlability intelligence.

Iqbal R

Robots.txt & Sitemap Discovery API

213x/robots-sitemap-discovery

Discover robots.txt rules, sitemap URLs, crawl delays, and basic crawlability signals for domains or website URLs.

Sitemap Indexability & AI Crawler Audit API

dobus/sitemap-indexability-ai-crawler-audit

Audit sitemap URLs, robots.txt, HTTP status, canonical/noindex signals, and AI crawler access as structured JSON for technical SEO and agents.

Afonso Fernandes

AI Readiness Auditor

muhammadafzal/ai-readiness-auditor

Audit website crawlability, AI bot access, llms.txt, sitemaps, metadata, JSON-LD, content, and accessibility with prioritized page-level fixes.

Muhammad Afzal

Store/Crawler Surface Preflight

checksmithcats/store-crawler-surface-preflight

Cross-check public robots.txt, sitemap, noindex, llms.txt, policy links, and contact signals from a storefront URL. Reads only public URLs; no account access.

Checksmith Cats

Indexability Audit

zerobreak/indexability-audit

Indexability audit tool that checks robots.txt, meta robots tags, X-Robots-Tag headers, and canonical URLs for any list of pages, so SEO teams know which ones Google can actually crawl and index.

ZeroBreak

Technical SEO Audit & Indexability Checker

getascraper/seo-release-indexability-guard

Use as a website SEO checker for public and staging URLs. Run a technical SEO audit for indexability, noindex, robots.txt, canonical tags, redirects, meta tags, sitemaps, internal links, and structured data. Get prioritized fixes and change tracking. No login or API token required. Pay per event.