# AI Search Readiness Audit: GEO, llms.txt & AI Crawlers (`iveeeeeen/ai-search-readiness-audit`) Actor

Bulk-audit websites for visibility in ChatGPT, Claude, Perplexity, Gemini and Google AI Overviews: AI crawler access for 19 bots, llms.txt, sitemap, schema.org JSON-LD, server-rendered content, metadata. 0-100 score, prioritized fixes, ready-to-paste llms.txt.

- **URL**: https://apify.com/iveeeeeen/ai-search-readiness-audit.md
- **Developed by:** [Ivan Alvarado](https://apify.com/iveeeeeen) (community)
- **Categories:** SEO tools, AI, Marketing
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $50.00 / 1,000 site auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Search Readiness Audit: GEO/AEO, llms.txt & AI Crawler Checker

**Find out if ChatGPT, Claude, Perplexity, Gemini and Google AI Overviews can find, read and cite your website, then get a prioritized fix list.** Bulk-audit 1 or 10,000 sites in one run. You pay per site audited, and failed sites are free.

### What does AI Search Readiness Audit do?

For every website you give it, this Actor checks the signals that decide whether AI answer engines can use your content (Generative Engine Optimization / Answer Engine Optimization):

| Category (weight) | What is checked |
|---|---|
| **AI crawler access (30%)** | robots.txt rules for **19 AI and search bots**: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Googlebot, Bingbot, Applebot-Extended, DuckAssistBot, Meta-ExternalAgent, Amazonbot, CCBot, Bytespider, cohere-ai, MistralAI-User. Also flags homepage noindex and WAF/bot-protection blocks (403s) |
| **Discovery (20%)** | `/llms.txt` and `/llms-full.txt` (llmstxt.org format check), XML sitemap, sitemap declared in robots.txt, markdown alternates |
| **Structured data (20%)** | schema.org JSON-LD types per page, Organization/Person entity, FAQPage/HowTo, invalid JSON-LD |
| **Content extractability (20%)** | Server-rendered text vs JavaScript-only shells (most AI crawlers don't run JS), H1/heading outline, visible dates |
| **Metadata & trust (10%)** | title, meta description, canonical, `lang`, Open Graph, author signals, HTTP errors |

Each site gets a **0-100 score, an A-F grade, category scores, and prioritized issues (high/medium/low), each with a concrete fix**. When something is missing you also get **a ready-to-paste `llms.txt`** and **a robots.txt snippet** that unblocks the AI search bots.

### Who is it for?

- **SEO and GEO agencies** running AI-visibility audits for clients and prospects at scale
- **Lead generation**: score a list of prospect domains, then pitch the ones graded D/F
- **Marketing and content teams** tracking AI-readiness across their own properties (schedule it weekly)
- **AI agents and MCP clients** that need a fast, structured website-readiness check

### How to use it

1. Enter one or more websites (domains or URLs).
2. Optionally raise **Pages to sample per site** (the homepage plus diverse pages from the sitemap; default 5).
3. Click **Start**, then download results as JSON, CSV, Excel, or through the API.

#### Input example

```json
{
  "urls": ["https://apify.com", "python.org", "example.com"],
  "maxPagesPerSite": 5
}
```

#### Output example (trimmed)

```json
{
  "site": "https://example.com",
  "score": 44,
  "grade": "F",
  "categoryScores": { "aiCrawlerAccess": 95, "discovery": 0, "structuredData": 0, "contentExtractability": 52, "metadataAndTrust": 55 },
  "issues": [
    { "priority": "high", "category": "discovery", "issue": "No XML sitemap discovered (robots.txt Sitemap: line or /sitemap.xml).", "fix": "Publish /sitemap.xml and reference it from robots.txt with a Sitemap: line." },
    { "priority": "high", "category": "structured-data", "issue": "No JSON-LD structured data on any sampled page.", "fix": "Add schema.org JSON-LD (WebSite, Organization, Article/Product as relevant)." }
  ],
  "aiCrawlerAccess": [ { "token": "OAI-SearchBot", "operator": "OpenAI", "purpose": "search", "allowed": true, "rule": "wildcard" } ],
  "llmsTxt": { "present": false },
  "sitemap": { "urlsFound": 0, "sitemapsChecked": 1, "declaredInRobots": false },
  "pagesAudited": 1,
  "recommendedLlmsTxt": "# Example Domain\n\n## Key pages\n\n- [Example Domain](https://example.com/)\n"
}
```

Each result also has a `pages` array with per-page signals: word count, JSON-LD types, H1 count, canonical, Open Graph, noindex, response time, and more.

### Pricing (pay per event)

- **$0.05 per site audited** (covers the homepage plus up to 4 more pages)
- **$0.005 per extra page** when you set `maxPagesPerSite` above 5
- **Sites that fail to load are not charged.**

100 sites cost about $5. You can cap your spend with the run's maximum-charge setting, and the Actor stops cleanly when the cap is reached.

### Responsible crawling

The Actor reads only public pages: robots.txt, llms.txt, sitemaps, and a small sample of pages per site. It identifies itself as `AIReadinessAuditBot` and skips sampled pages that robots.txt disallows for it (configurable). It never logs in and never bypasses bot protection.

### FAQ

**Is a high score a guarantee of AI citations?** No. It measures technical readiness (can AI systems access, parse and trust your pages). Citations also depend on content quality and authority.

**Should I allow GPTBot / ClaudeBot (training bots)?** That's a business decision, so blocking them does not lower your score. The score only penalizes blocking *search* and *user-triggered* bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot, ChatGPT-User…), because blocking those removes you from AI answers.

**Can I run it on a schedule?** Yes. Use Apify Schedules to re-audit weekly and track score changes.

**Found a bug or need a check added?** Open an issue on the Actor's Issues tab.

# Actor input Schema

## `urls` (type: `array`):

Domains or URLs (one per line). Each site is one audit. Example: apify.com, https://www.example.com/blog

## `maxPagesPerSite` (type: `integer`):

Homepage plus diverse pages picked from the XML sitemap. Up to 5 pages are included in the site-audit price; pages 6-25 are billed as extra-page events.

## `respectRobotsTxt` (type: `boolean`):

Skip sampled pages that robots.txt disallows for this auditor (the homepage, robots.txt, llms.txt and sitemap are always checked).

## `includePageDetails` (type: `boolean`):

Include the full per-page signal breakdown in each result. Turn off for compact output.

## `concurrency` (type: `integer`):

How many sites to audit at the same time.

## Actor input object example

```json
{
  "urls": [
    "https://apify.com",
    "https://www.python.org"
  ],
  "maxPagesPerSite": 5,
  "respectRobotsTxt": true,
  "includePageDetails": true,
  "concurrency": 3
}
```

# Actor output Schema

## `overview` (type: `string`):

All audit results (default view: score, grade, category scores, prioritized fixes).

## `aiCrawlers` (type: `string`):

Per-site robots.txt access for AI/search bots, llms.txt and sitemap status, suggested llms.txt.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://apify.com",
        "https://www.python.org"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("iveeeeeen/ai-search-readiness-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://apify.com",
        "https://www.python.org",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("iveeeeeen/ai-search-readiness-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://apify.com",
    "https://www.python.org"
  ]
}' |
apify call iveeeeeen/ai-search-readiness-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,iveeeeeen/ai-search-readiness-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7ovoVv2HND3SXToOf/builds/KJ5ZVf7gMNwsPdrEg/openapi.json