# AI Agent Readiness Audit — llms.txt, robots.txt & Schema (`paoe/ai-agent-readiness-audit`) Actor

Check whether AI agents can use a website: robots.txt AI-agent blocks, llms.txt, JSON-LD structured data, sitemap and summarisation signals.

- **URL**: https://apify.com/paoe/ai-agent-readiness-audit.md
- **Developed by:** [Rashad Flet](https://apify.com/paoe) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $30.00 / 1,000 site auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Check whether AI agents can actually use a website. Audits robots.txt for AI-agent blocking (GPTBot, ClaudeBot, PerplexityBot and more), llms.txt presence, JSON-LD Product/Organization structured data, sitemap availability and summarisation signals. Returns a 0-100 score, a letter grade and a named finding for every deduction, so the result is an actionable fix list, not just a number. Built by PAOE.

Each item is processed individually and charged as its own event, so you pay only for what the run actually delivers. Results are written to the run's dataset as one JSON object per item, ready to download as JSON, CSV or Excel, or to pull through the Apify API.

### What it checks

For each URL: whether robots.txt blocks named AI agents (GPTBot, ClaudeBot, PerplexityBot and others), whether an llms.txt file is published, whether the page carries JSON-LD `Organization` or `Product` structured data, whether a sitemap is advertised, and whether the page carries the summarisation signals assistants use (a meta description and a clear heading).

### Pricing

| Event | What it covers | Price |
| --- | --- | --- |
| `site-audited` | site audited (primary) | $0.05 per event |

Volume tiers reduce the price automatically on higher Apify plans: Bronze 15% off, Silver 25% off and Gold or above 40% off the listed free-tier price. The charge is per item processed, not per run.

### Input

| Field | Type | Required | Default | Description |
| --- | --- | --- | --- | --- |
| `urls` | array | yes | `["https://example.com"]` | One or more site URLs. Each URL is one audited site and one charged event. |
| `maxUrls` | integer | no | `10` | Safety cap on how many URLs are audited in one run. |

Example input:

```json
{
  "urls": [
    "https://example.com"
  ],
  "maxUrls": 10
}
```

### Output

One JSON object per item in the run's dataset. Every result carries the input it came from plus the fields this Actor measures, so the output can be joined back to your own data without guessing which row is which. The final dataset entry is a `summary` object with the run's totals.

### Typical use cases

- Check whether AI crawlers and answer engines can actually read a site.
- Prepare a site for AI-agent traffic before competitors do.
- Produce a prioritised, per-item fix list for a web or SEO team.

### Limitations

The sitemap is checked for availability and reported as a finding, but its URLs are not added automatically. Audited URLs come from the `urls` input, so you control exactly what is checked and charged. Sites behind a login, a paywall or an aggressive bot wall return unusable content and are reported as errors, not counted as successes.

#### Does it follow the sitemap?

Every limitation above is reported per item in a `findings` entry with a `level` of `fail`, `warn`, `info` or `ok`, a machine-readable `code` and a concrete `action`. If something cannot be checked it is reported as such rather than assumed to be fine.

#### Can it run on a schedule?

Yes. Save a task from this Actor with your inputs, then set a schedule on the task. Scheduled runs recur with the same inputs, which is the intended way to use it for ongoing monitoring.

#### Can I audit a whole site?

Pass the pages you want checked in `urls`, up to `maxUrls`. Every URL audited is one charged event, so the cost is exactly the number of pages you list.

### Notes

If a site returns something unexpected, open an issue on the Actor's page with the URL and the input used, and it will be looked at.

### Related Actors

- [Bulk Broken Link & Redirect Audit](https://apify.com/paoe/bulk-broken-link-redirect-audit)

### Keywords

ai agent readiness, llms.txt, robots.txt, GPTBot, JSON-LD, structured data, SEO audit, answer engine optimization, AEO.

# Actor input Schema

## `urls` (type: `array`):

One or more site URLs. Each URL is one audited site and one charged event.

## `maxUrls` (type: `integer`):

Safety cap on how many URLs are audited in one run.

## Actor input object example

```json
{
  "urls": [
    "https://example.com"
  ],
  "maxUrls": 10
}
```

# Actor output Schema

## `reports` (type: `string`):

One JSON report per audited site: score, grade and a named finding for every deduction.

## `summary` (type: `string`):

Totals across the run: sites audited, mean score and the counts behind them.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("paoe/ai-agent-readiness-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["https://example.com"] }

# Run the Actor and wait for it to finish
run = client.actor("paoe/ai-agent-readiness-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com"
  ]
}' |
apify call paoe/ai-agent-readiness-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,paoe/ai-agent-readiness-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/dC4owjUFwqPdZ6YUL/builds/dlX68cMovfbDGMART/openapi.json