Pricing

from $8.50 / 1,000 site testeds

Try for free

Go to Apify Store

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site?

Try for free

Tests whether AI search and answer bots (OpenAI, Anthropic, Perplexity, Google, Bing...) can actually read your site: robots.txt per bot (search vs training), a live fetch as each bot vs a browser to catch CDN/WAF blocks, JS-only content, llms.txt and entity schema, with fixes.

Pricing

from $8.50 / 1,000 site testeds

Rating

0.0

(0)

Developer

Centor Prat

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Categories

SEO tools

Developer tools

You can access the AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "urls": [
12        "react.dev"
13    ]
14};
15
16// Run the Actor and wait for it to finish
17const run = await client.actor("centorprat/ai-crawler-access-test").call(input);
18
19// Fetch and print Actor results from the run's dataset (if any)
20console.log('Results from dataset');
21console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
22const { items } = await client.dataset(run.defaultDatasetId).listItems();
23items.forEach((item) => {
24    console.dir(item);
25});
26
27// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

AI Crawler Access Test - GPTBot, ClaudeBot, PerplexityBot API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API in Python

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API through CLI

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? OpenAPI definition

AI Crawler Access Test - Can ChatGPT & AI Bots Read Your Site? API

Axe Accessibility Tester

hanamira/axe-accessibility-tester

Test any website for WCAG accessibility issues using axe-core. Scan single pages or crawl entire sites. Get detailed reports with violation severity, affected elements, and remediation guidance. Supports WCAG 2.0, 2.1, and 2.2 at A, AA, and AAA levels.

hannah mira

Dead-Link Watchdog

practicaltools/dead-link-crawler

Automatically monitor your website for broken, redirected, or slow links. Configure which HTTP status codes to flag, schedule recurring scans, and get structured reports — all for just $0.0008 per link. Keep your site clean, fast, and error-free.

Practical Tools

Robots AI Crawler Policy Checker

seeb/robots-ai-crawler-policy-checker

Analyze robots.txt and crawler policy text for AI bot access rules, disallow paths, sitemap references, and blocked model-training crawlers.

Techionik

Web Crawler & Semantic Schema-Enhanced Extractor

devil_port369-owner/web-crawler

Depth-controlled web crawler that transforms websites into structured analytics-ready data. Starting from one or more URLs, it crawls internal links up to a configurable depth and outputs detailed JSON records per page

DataFusionX

5.0

(8)

AI Crawler Access Checker: GPTBot, ClaudeBot, robots.txt

atalaya-ai/ai-crawler-access-checker

Bulk-check which AI crawlers (GPTBot, ClaudeBot, PerplexityBot...) each domain allows or blocks in robots.txt. Pay per domain, EN/ES, no API keys.

Atalaya AI

Amazon Listing Audit - Listing Score, Reviews & Competitor Gaps

centorprat/amazon-listing-audit

Audit Amazon listings by ASIN: listing-content score with prioritised fixes (title, bullets, images, video, A+), review health (4-star cliff, 1-star share, polarized-rating pattern), buy box, stock, BSR and demand, plus a benchmark vs the competitors you list.

Centor Prat

Bulk AI Crawler Access Checker

srvscripts/ai-crawler-access-checker

Audit AI crawler access across many domains at once. For each domain: robots.txt rules for GPTBot, ClaudeBot, PerplexityBot, CCBot and more, plus llms.txt, ai.txt, RSL licensing — and a live User-Agent test that catches domains blocking bots their robots.txt actually allows.

srvScripts

Amazon Book & T-Shirt Niche Analyzer

vncntlabs/amazon-book-tshirt-niche-analyzer

Amazon niche research for low-content books, Kindle and print-on-demand t-shirts. One keyword in, one report out: Amazon search volume, top-20 price and review spread, Best Sellers Rank, publish dates, self-publisher share and a USPTO trademark screen. Pay per niche.

VNCNT Labs

LinkedIn Company Scraper - Profile, Posts, Similar Companies

centorprat/linkedin-company-scraper

LinkedIn company data from URLs, names or domains: description, website, industry, size, employees, HQ, founded, specialties, followers, locations, similar companies and recent posts with likes and comments. No login, no personal profiles.

Centor Prat

Website Audit & AI Readiness Checker

professional_automation_tools1/website-audit-ai-readiness

AI readiness score per site and SEO score per page: access of 20 AI crawlers (ChatGPT, Claude, Perplexity, Google, Bing), content visible without JavaScript, structured data, nosnippet, sitemap, llms.txt. Fix hints and HTML report.