Pricing

from $8.50 / 1,000 site testeds

Try for free

Go to Apify Store

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site?

Try for free

Tests whether AI search and answer bots (OpenAI, Anthropic, Perplexity, Google, Bing...) can actually read your site: robots.txt per bot (search vs training), a live fetch as each bot vs a browser to catch CDN/WAF blocks, JS-only content, llms.txt and entity schema, with fixes.

Pricing

from $8.50 / 1,000 site testeds

Rating

0.0

(0)

Developer

Centor Prat

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

AI Crawler Access Test - GPTBot, ClaudeBot, PerplexityBot API through CLI

The Apify CLI is the official tool that allows you to use AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? locally, providing convenience functions and automatic retries on errors.

Install Apify CLI

Using installation script (macOS/Linux):

$curl -fsSL https://apify.com/install-cli.sh | bash

Using installation script (Windows):

$irm https://apify.com/install-cli.ps1 | iex

Using Homebrew:

$brew install apify-cli

Using npm:

$npm install -g apify-cli

Other API clients include:

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API in Python

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API in JavaScript

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? OpenAPI definition

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API

Axe Accessibility Tester

hanamira/axe-accessibility-tester

Test any website for WCAG accessibility issues using axe-core. Scan single pages or crawl entire sites. Get detailed reports with violation severity, affected elements, and remediation guidance. Supports WCAG 2.0, 2.1, and 2.2 at A, AA, and AAA levels.

hannah mira

Dead-Link Watchdog

practicaltools/dead-link-crawler

Automatically monitor your website for broken, redirected, or slow links. Configure which HTTP status codes to flag, schedule recurring scans, and get structured reports — all for just $0.0008 per link. Keep your site clean, fast, and error-free.

Practical Tools

Robots AI Crawler Policy Checker

seeb/robots-ai-crawler-policy-checker

Analyze robots.txt and crawler policy text for AI bot access rules, disallow paths, sitemap references, and blocked model-training crawlers.

Techionik

Web Crawler & Semantic Schema-Enhanced Extractor

devil_port369-owner/web-crawler

Depth-controlled web crawler that transforms websites into structured analytics-ready data. Starting from one or more URLs, it crawls internal links up to a configurable depth and outputs detailed JSON records per page

DataFusionX

5.0

(8)

AI Crawler Access Checker: GPTBot, ClaudeBot, robots.txt

atalaya-ai/ai-crawler-access-checker

Bulk-check which AI crawlers (GPTBot, ClaudeBot, PerplexityBot...) each domain allows or blocks in robots.txt. Pay per domain, EN/ES, no API keys.

Atalaya AI

Amazon Listing Audit - Listing Score, Reviews & Competitor Gaps

centorprat/amazon-listing-audit

Audit Amazon listings by ASIN: listing-content score with prioritised fixes (title, bullets, images, video, A+), review health (4-star cliff, 1-star share, polarized-rating pattern), buy box, stock, BSR and demand, plus a benchmark vs the competitors you list.

Centor Prat

Bulk AI Crawler Access Checker

srvscripts/ai-crawler-access-checker

Audit AI crawler access across many domains at once. For each domain: robots.txt rules for GPTBot, ClaudeBot, PerplexityBot, CCBot and more, plus llms.txt, ai.txt, RSL licensing — and a live User-Agent test that catches domains blocking bots their robots.txt actually allows.

srvScripts

Amazon Book & T-Shirt Niche Analyzer

vncntlabs/amazon-book-tshirt-niche-analyzer

Amazon niche research for low-content books, Kindle and print-on-demand t-shirts. One keyword in, one report out: Amazon search volume, top-20 price and review spread, Best Sellers Rank, publish dates, self-publisher share and a USPTO trademark screen. Pay per niche.

VNCNT Labs

LinkedIn Company Scraper - Profile, Posts, Similar Companies

centorprat/linkedin-company-scraper

LinkedIn company data from URLs, names or domains: description, website, industry, size, employees, HQ, founded, specialties, followers, locations, similar companies and recent posts with likes and comments. No login, no personal profiles.

Centor Prat

Website Audit & AI Readiness Checker

professional_automation_tools1/website-audit-ai-readiness

AI readiness score per site and SEO score per page: access of 20 AI crawlers (ChatGPT, Claude, Perplexity, Google, Bing), content visible without JavaScript, structured data, nosnippet, sitemap, llms.txt. Fix hints and HTML report.