# GEO / AEO Website Audit — AI Search Readiness (`rtworule/ai-search-readiness-auditor`) Actor

Audit public websites for GEO and AEO readiness: AI crawler access, robots.txt, llms.txt, sitemaps, Schema.org, social metadata, scores, and prioritized fixes.

- **URL**: https://apify.com/rtworule/ai-search-readiness-auditor.md
- **Developed by:** [Kunteper Koyu](https://apify.com/rtworule) (community)
- **Categories:** SEO tools, AI, Developer tools
- **Stats:** 2 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 site auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## GEO / AEO Website Audit — AI Search Readiness

Audit public pages for the technical signals used by AI search and answer engines. Get a practical 0–100 score, crawler policy checks, structured-data findings, and prioritized fixes for generative engine optimization (GEO), answer engine optimization (AEO), and technical SEO.

### What you get

- HTTP status, redirects, title, meta description, canonical URL, Open Graph, and Twitter metadata
- Schema.org / JSON-LD detection with discovered `@type` values
- `robots.txt` policy checks for GPTBot, ChatGPT-User, PerplexityBot, ClaudeBot, and Google-Extended
- sitemap and `/llms.txt` discovery
- a score, letter grade, and actionable findings for every URL
- one clean dataset row per audited page, ready for export or API use

### Common use cases

- audit a site before a GEO/AEO consulting engagement
- find pages that AI crawlers cannot access
- prioritize missing schema, metadata, sitemap, and `llms.txt` work
- monitor client or portfolio sites on a schedule
- enrich SEO dashboards, reports, and content inventories

### Quick start

1. Click **Try for free** or **Run**.
2. Add one or more public page URLs.
3. Keep the default checks enabled and start the run.
4. Open **Dataset** to view, filter, or export the results.

```json
{
  "urls": ["https://example.com", "https://apify.com"],
  "concurrency": 10,
  "timeoutSeconds": 20,
  "includeSitemapCheck": true,
  "includeLlmsTxtCheck": true
}
````

### Example result

```json
{
  "inputUrl": "https://example.com",
  "finalUrl": "https://example.com/",
  "statusCode": 200,
  "ok": true,
  "score": 82,
  "grade": "B",
  "title": "Example Domain",
  "canonicalUrl": "https://example.com/",
  "hasSchemaOrg": true,
  "schemaTypes": ["Organization"],
  "hasSitemap": true,
  "hasLlmsTxt": false,
  "aiCrawlerPolicy": {
    "GPTBot": "allowed",
    "ChatGPT-User": "unknown",
    "PerplexityBot": "unknown",
    "ClaudeBot": "unknown",
    "Google-Extended": "unknown"
  },
  "findings": [{
    "code": "LLMS_TXT_NOT_FOUND",
    "severity": "info",
    "message": "llms.txt was not found at the site root."
  }],
  "checkedAt": "2026-07-09T03:00:00.000Z"
}
```

### Run by API

Set `APIFY_TOKEN` in your environment. Replace it with a secret-manager reference in production; never commit it.

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/rtworule~ai-search-readiness-auditor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://example.com"],"includeSitemapCheck":true,"includeLlmsTxtCheck":true}'
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('rtworule/ai-search-readiness-auditor').call({
  urls: ['https://example.com'],
  includeSitemapCheck: true,
  includeLlmsTxtCheck: true,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("rtworule/ai-search-readiness-auditor").call(run_input={
    "urls": ["https://example.com"],
    "includeSitemapCheck": True,
    "includeLlmsTxtCheck": True,
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)
```

### Pricing

This Actor uses pay per event. A `site-audited` event costs **$0.001 per stored result**: 100 audited pages cost $0.10 and 1,000 cost $1.00 in event charges. Apify applies a **$0.01 minimum run charge**. Failed inputs that cannot produce a stored diagnostic row are not charged as a result event. The Actor respects the run's maximum total charge.

### Integrations and automation

- schedule weekly audits in Apify and send dataset results to a webhook
- connect Make, Zapier, or n8n through the Apify integration or API
- export CSV/XLSX for client reports, Looker Studio, Sheets, or a data warehouse
- trigger a Slack or email alert when `score`, `grade`, or crawler policy changes
- use deterministic URL fields to compare current results with an earlier dataset

### FAQ and troubleshooting

**Does this guarantee AI citations or rankings?** No. The score measures technical readiness signals; rankings and citations depend on content quality, authority, relevance, and engine behavior.

**Why is metadata missing even though I see it in a browser?** This is a fast HTTP/static-HTML audit. Metadata rendered only by JavaScript is not evaluated.

**Why was a URL rejected?** Only public HTTP/HTTPS destinations are accepted. URLs with credentials and local, private, reserved, or metadata-service destinations are blocked for safety.

**Why is a crawler policy `unknown`?** The site's `robots.txt` may not define a rule for that crawler, or the file may not be available.

**A run timed out. What should I change?** Reduce concurrency for a slow host or increase `timeoutSeconds` up to 60.

### Responsible use, limitations, and support

The Actor reads user-provided public pages and same-origin root files (`robots.txt`, `sitemap.xml`, and `llms.txt`). It does not log in, bypass access controls, or collect private data. The HTML sample is capped at 256 KiB. Sitemap discovery does not crawl every sitemap URL. Users are responsible for the URLs they submit and their downstream use.

If a valid public page produces an unexpected result, open an issue from the Actor page with the run ID, affected URL, and expected behavior. Do not include credentials or sensitive data.

### More tools from this developer

- [Bulk URL Status & Health Checker](https://apify.com/rtworule/bulk-url-health-auditor)
- [Website Tech Stack Detector](https://apify.com/rtworule/bulk-website-tech-detector)
- [RSS, Atom & JSON Feed Normalizer](https://apify.com/rtworule/public-feed-normalizer)
- [Greenhouse, Lever & Ashby Jobs Normalizer](https://apify.com/rtworule/public-ats-job-feed-normalizer)

# Actor input Schema

## `urls` (type: `array`):

Public HTTP or HTTPS pages to inspect. Private and local network destinations are rejected.

## `concurrency` (type: `integer`):

Number of URLs checked in parallel.

## `timeoutSeconds` (type: `integer`):

Maximum seconds allowed for each request.

## `includeSitemapCheck` (type: `boolean`):

Look for a sitemap in robots.txt and at /sitemap.xml.

## `includeLlmsTxtCheck` (type: `boolean`):

Look for /llms.txt at the site root.

## Actor input object example

```json
{
  "urls": [
    "https://example.com"
  ],
  "concurrency": 10,
  "timeoutSeconds": 20,
  "includeSitemapCheck": true,
  "includeLlmsTxtCheck": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("rtworule/ai-search-readiness-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("rtworule/ai-search-readiness-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call rtworule/ai-search-readiness-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=rtworule/ai-search-readiness-auditor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "GEO / AEO Website Audit — AI Search Readiness",
        "description": "Audit public websites for GEO and AEO readiness: AI crawler access, robots.txt, llms.txt, sitemaps, Schema.org, social metadata, scores, and prioritized fixes.",
        "version": "0.1",
        "x-build-id": "loDoJGDaBGLAmkjw9"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/rtworule~ai-search-readiness-auditor/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-rtworule-ai-search-readiness-auditor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/rtworule~ai-search-readiness-auditor/runs": {
            "post": {
                "operationId": "runs-sync-rtworule-ai-search-readiness-auditor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/rtworule~ai-search-readiness-auditor/run-sync": {
            "post": {
                "operationId": "run-sync-rtworule-ai-search-readiness-auditor",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "urls"
                ],
                "properties": {
                    "urls": {
                        "title": "URLs",
                        "type": "array",
                        "description": "Public HTTP or HTTPS pages to inspect. Private and local network destinations are rejected.",
                        "default": [
                            "https://example.com"
                        ],
                        "items": {
                            "type": "string"
                        }
                    },
                    "concurrency": {
                        "title": "Concurrency",
                        "minimum": 1,
                        "maximum": 50,
                        "type": "integer",
                        "description": "Number of URLs checked in parallel.",
                        "default": 10
                    },
                    "timeoutSeconds": {
                        "title": "Request timeout",
                        "minimum": 1,
                        "maximum": 60,
                        "type": "integer",
                        "description": "Maximum seconds allowed for each request.",
                        "default": 20
                    },
                    "includeSitemapCheck": {
                        "title": "Check sitemap",
                        "type": "boolean",
                        "description": "Look for a sitemap in robots.txt and at /sitemap.xml.",
                        "default": true
                    },
                    "includeLlmsTxtCheck": {
                        "title": "Check llms.txt",
                        "type": "boolean",
                        "description": "Look for /llms.txt at the site root.",
                        "default": true
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```