# AI SHOPPING READINESS AUDITOR — why agents skip your products (`egeusta/ai-readiness-auditor`) Actor

Audit product pages for AI shopping readiness. Score schema.org Product and Offer data, find missing fields with evidence, and see which AI crawlers robots.txt allows.

- **URL**: https://apify.com/egeusta/ai-readiness-auditor.md
- **Developed by:** [Ege](https://apify.com/egeusta) (community)
- **Categories:** E-commerce, SEO tools, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.50 / 1,000 product page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

**Find out why AI shopping assistants can—or cannot—use your product pages.**

AI Shopping Readiness Auditor checks the machine-readable product information on public e-commerce pages. It gives every successfully fetched product a 0–100 readiness score, names the exact schema.org fields that are missing, preserves a compact evidence excerpt, and reports which AI crawlers the store allows in `robots.txt`.

Use it for catalog QA, technical SEO, agentic-commerce preparation, migration checks, and prioritized developer work lists. It measures observable page signals; it does not claim to predict whether ChatGPT, Gemini, Claude, Perplexity, or another service will recommend or rank a product.

### Quick start

Audit one or more product pages directly:

```json
{
  "startUrls": [
    { "url": "https://shop.example/products/legacy-tee" }
  ],
  "discoverProducts": false,
  "maxProducts": 25,
  "checkCrawlerAccess": true,
  "onlyProblems": false
}
```

To audit a store, provide its homepage, enable `discoverProducts`, and set a sensible `maxProducts`. The Actor reads `robots.txt`, follows declared XML sitemaps—including gzip-compressed sitemaps—and identifies likely product URLs.

### What each product audit checks

The score separates fields required to list and price a product from fields that help an assistant compare and trust it.

Required signals carry most of the score:

- Product name and brand
- SKU, GTIN, UPC, EAN, MPN, or another stable identifier
- Offer price and price currency
- Availability
- Product image

Advisory signals improve comparison quality:

- Product description
- Return policy
- Shipping details
- Aggregate rating

A product missing a price is not treated as “almost ready.” It is missing information an automated shopper needs before it can compare the item responsibly.

### Evidence, not guesses

Every paid `audit` row can include the trimmed Product markup that was evaluated. The `present`, `missing`, and `advisory` arrays make the result easy to turn into a ticket or QA checklist.

If a page cannot be fetched, it is returned as `unreachable`, is not graded, and is not charged. A timeout or HTTP block describes the request outcome—not the quality of the merchant’s catalog.

### AI crawler access

When `checkCrawlerAccess` is enabled, the Actor adds one free `access` row per store. It separates crawlers used for search or user-requested retrieval from crawlers associated with model training.

The output includes:

- `allowed`
- `blockedSearch`
- `blockedTraining`
- `unknown`
- `robotsStatus`

A missing `robots.txt` file and an unreachable one are not treated as the same condition. A 404 means no rules were published; a failed request remains unknown.

### Input

| Field | Description |
|---|---|
| `startUrls` | Product page URLs, or store homepages when discovery is enabled |
| `discoverProducts` | Discover product pages from robots.txt and XML sitemaps |
| `maxProducts` | Maximum number of product pages considered in the run |
| `checkCrawlerAccess` | Add the free per-store AI crawler access row |
| `onlyProblems` | Omit ready pages; omitted pages are not charged |
| `crawlerIdentity` | Optional custom User-Agent for your organization |

Start with 10–25 pages. Review the results, then increase `maxProducts` for a full catalog run.

### Output

The default dataset contains three row types in one comprehensive **Audit results** view. Use `rowType` to distinguish them:

#### Product page audits

```json
{
  "rowType": "audit",
  "ok": true,
  "url": "https://shop.example/products/legacy-tee",
  "name": "Legacy Fitted T-Shirt",
  "score": 63,
  "verdict": "partial",
  "noMarkup": false,
  "present": ["name", "brand", "price", "priceCurrency"],
  "missing": ["sku_or_gtin", "availability"],
  "advisory": ["returnPolicy", "shippingDetails"],
  "source": "jsonld",
  "evidence": "{"@type":"Product","name":"Legacy Fitted T-Shirt"..."
}
```

#### AI crawler access

One free row per store with `robotsStatus`, allowed crawlers, blocked search crawlers, blocked training crawlers, and unknown results.

#### Unreachable pages

Free, ungraded rows with URL, HTTP status, error message, and timestamp.

### Score interpretation

| Score | Verdict | Meaning |
|---|---|---|
| 90–100 | `ready` | The page exposes the core product and offer signals |
| 60–89 | `partial` | The page is usable but has material gaps |
| 0–59 | `not-ready` | Required product or offer information is missing |

Read `noMarkup` separately. “No Product node exists” and “the Product node is incomplete” usually require different fixes.

### Pricing

Pay per event:

- **$0.01 per Actor start** at the default 512 MB memory
- **$0.0025 per delivered product-page audit** — $2.50 per 1,000
- AI crawler access rows are free
- Unreachable pages are free
- Pages omitted by `onlyProblems` are free
- Apify platform usage is included

Examples before any user-level pricing discounts:

| Delivered audits | Estimated price |
|---:|---:|
| 10 | $0.035 |
| 100 | $0.26 |
| 1,000 | $2.51 |

The Actor respects the maximum total charge configured for the run and stops before continuing unpaid work.

### Reliability and politeness

Requests to one host are spaced out, response sizes and timeouts are capped, and HTTP 429 receives one backed-off retry. The Actor reads public pages only and does not log in, solve challenges, bypass access controls, or submit purchases.

### Limitations

- No browser rendering: product data available only after client-side JavaScript may be absent from the initial HTML.
- Scores are deterministic technical triage, not certification or a guarantee of placement in any AI answer.
- Crawler access is evaluated at the site root; path-specific rules may differ.
- Results are a point-in-time snapshot and can change when page markup or robots.txt changes.
- Product discovery depends on public robots.txt, sitemaps, and fallback homepage links. It cannot find intentionally hidden catalog pages.

### Automation

Run the Actor on an Apify schedule for recurring catalog QA, call it through the Apify API, or connect the structured dataset to a spreadsheet, database, webhook, or downstream Actor.

### Responsible use and support

Audit only public pages you are permitted to access, and follow applicable website terms, laws, and organizational policies. The output is technical information, not legal, regulatory, or commercial advice.

For support, open an Actor issue with a sample URL, the run ID, and the dataset row that looked unexpected. Never include credentials or private customer data.

# Actor input Schema

## `startUrls` (type: `array`):

Add product pages directly. To audit a whole store, add its homepage and enable Find product pages.

## `discoverProducts` (type: `boolean`):

Treat each URL as a store and discover product pages from robots.txt and declared sitemaps.

## `maxProducts` (type: `integer`):

Hard cap on the number of product pages audited in this run.

## `checkCrawlerAccess` (type: `boolean`):

Report which AI search, user-request, and training crawlers robots.txt permits. One free check per store.

## `onlyProblems` (type: `boolean`):

Do not save or charge pages graded ready. Use this to produce a focused remediation list.

## `crawlerIdentity` (type: `string`):

Optional crawler identity. Include a real contact when your policy requires it.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"
    }
  ],
  "discoverProducts": false,
  "maxProducts": 25,
  "checkCrawlerAccess": true,
  "onlyProblems": false
}
```

# Actor output Schema

## `results` (type: `string`):

The default dataset. Use rowType to distinguish audit, access, and unreachable records.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("egeusta/ai-readiness-auditor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/" }] }

# Run the Actor and wait for it to finish
run = client.actor("egeusta/ai-readiness-auditor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/"
    }
  ]
}' |
apify call egeusta/ai-readiness-auditor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,egeusta/ai-readiness-auditor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/TqS2vMsmOHmnoDJ0j/builds/gROr3ouEdQcobTDkU/openapi.json