# AI Agent Readiness Audit — llms.txt, AI Crawlers & AI Shopping (`signalloom/agent-readiness-audit`) Actor

Check if a website or store is ready for AI assistants and agents: AI crawler access in robots.txt (GPTBot, ClaudeBot, PerplexityBot…), llms.txt, readability without JavaScript, product structured data, price and stock, agentic checkout. Returns a 0–100 score and fix list. $0.03 per site.

- **URL**: https://apify.com/signalloom/agent-readiness-audit.md
- **Developed by:** [Signalloom](https://apify.com/signalloom) (community)
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $30.00 / 1,000 site auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Agent Readiness Audit: llms.txt, AI Crawlers & AI Shopping

More and more shoppers ask **ChatGPT, Claude, Perplexity and Gemini** what to buy. Those assistants can only recommend what they can **read**. This Actor checks a store or website the way an AI assistant sees it and gives it a **0–100 agent-readiness score**, a grade and a **prioritized fix list**.

- **AI crawler access**: robots.txt rules for 11 AI crawlers and agents (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Google-Extended, Applebot-Extended, CCBot…), split into **search and assistant** bots (they affect whether you're recommended) and **training** bots (a policy choice)
- **llms.txt**: whether the site offers the plain-text guide AI assistants use to understand it
- **Readable without JavaScript**: most AI crawlers don't run JavaScript, so a page built only in the browser looks empty to them
- **Product data on real product pages**: Product structured data, **machine-readable price, currency and stock**, GTIN/SKU, brand and reviews (it samples real product pages and skips category and marketing pages)
- **Identity and discovery**: Organization/Store structured data, XML sitemap
- **Agentic checkout signal**: whether the store advertises an agentic-commerce discovery link (such as Shopify's UCP), so AI agents can start a purchase
- Works on any platform: Shopify, WooCommerce, BigCommerce, custom. Sites that aren't stores get the general checks only.

**Price: $0.03 per site.** The prefilled example (2 sites) finishes in under a minute.

### Use this when…

- you want to know if **AI shopping assistants** (ChatGPT, Claude, Perplexity, Gemini) can read a store's products, prices and stock
- you need to **check llms.txt**: whether a site has one
- you want an **AI crawler audit**: which AI bots (GPTBot, ClaudeBot, PerplexityBot…) robots.txt allows or blocks
- you're doing **GEO / AI visibility** checks for many client or prospect sites at once
- you need a 0–100 **AI agent readiness** score and a prioritized fix list per site

### For AI agents

- **Run** `signalloom/agent-readiness-audit` (Apify MCP server `call-actor`, or the Apify API) with input like `{"websites": ["ooni.com"]}`
- **Returns** one JSON row per site in the default dataset: `agentReadinessScore` (0–100), `grade`, `hasLlmsTxt`, `aiCrawlers` (each bot allowed or blocked), `readableWithoutJs`, `productData` (price, stock, GTIN coverage), `agenticCheckoutSignal`, `issues` (fix list, errors first).
- **Typical cost:** 1 site = **$0.03**. 100 sites = $3.

### Who is it for?

- **E-commerce agencies**: a concrete, sellable "AI readiness" check for every client and prospect
- **Store owners and marketers**: find out whether AI assistants can see your products, prices and stock
- **SEO and GEO specialists**: generative-engine optimization audits at scale
- **SaaS and app makers**: find stores that need help with structured data or AI visibility

### How to use it

1. Add websites: `www.nutribullet.com`, `https://your-store.com`.
2. Optional: **Product pages to sample** (default 3).
3. Click **Start**. Sort by `agentReadinessScore` and work through each site's `issues`, which are sorted errors first.

**Recipe: agency prospecting.** Add 100 prospect stores, then filter to `grade` D or F, or to `issueCodes` containing `PRICE_NOT_MACHINE_READABLE` or `AI_SEARCH_BLOCKED`. Each one is a specific, fixable pitch.

### Output

One record per site (real example, shortened):

```json
{
  "domain": "nutribullet.com",
  "agentReadinessScore": 89,
  "grade": "A",
  "isStore": true,
  "hasLlmsTxt": false,
  "aiSearchCrawlersAllowed": true,
  "aiCrawlers": [
    { "agent": "OAI-SearchBot", "owner": "OpenAI", "purpose": "search", "allowed": true, "explicitRule": false }
  ],
  "readableWithoutJs": true,
  "structuredDataTypes": ["WebSite", "Organization"],
  "hasSitemap": true,
  "productPagesChecked": 3,
  "productData": { "productSchema": 1, "price": 1, "currency": 1, "availability": 1, "gtinOrSku": 1, "brand": 0, "reviews": 1 },
  "agenticCheckoutSignal": false,
  "issues": [
    { "severity": "warning", "code": "NO_LLMS_TXT", "message": "No /llms.txt: a plain-text guide that tells AI assistants what the site offers and where key pages are." },
    { "severity": "notice", "code": "NO_PRODUCT_BRAND", "message": "Brand missing from some product structured data." },
    { "severity": "notice", "code": "NO_AGENTIC_CHECKOUT_SIGNAL", "message": "No agentic-commerce discovery link (e.g. UCP), so AI agents get no standard way to start a checkout." }
  ]
}
```

`productData` values are the share of sampled product pages (0–1) that have each field. `isStore` is `null` when product pages exist but couldn't be read (for example, a maintenance or bot-check page). In that case the product checks are skipped and an issue explains why.

**Score:** 100 minus 15 per error, 7 per warning and 2 per notice. Grades: A ≥ 85, B ≥ 70, C ≥ 55, D ≥ 40, F below.

### Pricing

- **$0.03 per site** audited
- Sites that can't be loaded, are blocked by bot protection, or whose robots.txt asks crawlers not to visit: **free**

### Data handling

- Only public pages are read: the homepage, robots.txt, llms.txt, the sitemap and a few product pages. The Actor follows each site's robots.txt rules for general crawlers and never bypasses bot protection.
- No personal data is collected.

### Limitations

- It checks a sample of product pages (default 3), not the whole catalog.
- "Readable without JavaScript" is measured from the HTML the server sends. Some AI agents with a full browser can read more.
- AI crawler rules come from robots.txt only. Firewalls or bot protection that block AI bots at the network level aren't visible.
- Agentic-commerce standards are new and changing fast. The checks will be updated as they settle.

### FAQ

**How do I check if a website has an llms.txt file?**

Add the domain and run: `hasLlmsTxt` and `hasLlmsFullTxt` tell you, and a missing file shows up in the fix list. It checks a whole list of sites in one run.

**Is my robots.txt blocking GPTBot, ClaudeBot or PerplexityBot?**

`aiCrawlers` lists 11 AI crawlers and agents, whether each is allowed, and whether it is a search/assistant bot (affects being recommended) or a training bot (a policy choice).

**Can ChatGPT and other AI assistants see my store's prices and stock?**

The audit samples real product pages and reports in `productData` how many have machine-readable price, currency, availability, GTIN/SKU, brand and reviews. Missing ones become fix-list items.

**How do I track AI readiness over time?**

Schedule a monthly run on your own site or your clients' sites and compare `agentReadinessScore` and `issueCodes` between runs.

### Feedback

Want another check added? Open an issue on the **Issues** tab.

# Actor input Schema

## `websites` (type: `array`):

Store or website domains or URLs, one per line, e.g. "ooni.com". The homepage, robots.txt, llms.txt, sitemap and a few product pages are checked. Example: \["ooni.com", "https://www.nutribullet.com"].

## `maxProductPages` (type: `integer`):

How many product pages to check for structured data, price and stock (stores only). Example: 3.

## `maxConcurrency` (type: `integer`):

How many sites to audit at the same time.

## Actor input object example

```json
{
  "websites": [
    "www.nutribullet.com",
    "ooni.com"
  ],
  "maxProductPages": 3,
  "maxConcurrency": 3
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "websites": [
        "www.nutribullet.com",
        "ooni.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("signalloom/agent-readiness-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "websites": [
        "www.nutribullet.com",
        "ooni.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("signalloom/agent-readiness-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "websites": [
    "www.nutribullet.com",
    "ooni.com"
  ]
}' |
apify call signalloom/agent-readiness-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,signalloom/agent-readiness-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ca0xM9CASwwvgE8Na/builds/7TEO76c2tlW80dNUG/openapi.json