AI SHOPPING READINESS AUDITOR — why agents skip your products

Find out why AI shopping assistants can—or cannot—use your product pages.

AI Shopping Readiness Auditor checks the machine-readable product information on public e-commerce pages. It gives every successfully fetched product a 0–100 readiness score, names the exact schema.org fields that are missing, preserves a compact evidence excerpt, and reports which AI crawlers the store allows in robots.txt.

Use it for catalog QA, technical SEO, agentic-commerce preparation, migration checks, and prioritized developer work lists. It measures observable page signals; it does not claim to predict whether ChatGPT, Gemini, Claude, Perplexity, or another service will recommend or rank a product.

Quick start

Audit one or more product pages directly:

{
  "startUrls": [
    { "url": "https://shop.example/products/legacy-tee" }
  ],
  "discoverProducts": false,
  "maxProducts": 25,
  "checkCrawlerAccess": true,
  "onlyProblems": false
}

To audit a store, provide its homepage, enable discoverProducts, and set a sensible maxProducts. The Actor reads robots.txt, follows declared XML sitemaps—including gzip-compressed sitemaps—and identifies likely product URLs.

What each product audit checks

The score separates fields required to list and price a product from fields that help an assistant compare and trust it.

Required signals carry most of the score:

• Product name and brand
• SKU, GTIN, UPC, EAN, MPN, or another stable identifier
• Offer price and price currency
• Availability
• Product image

Advisory signals improve comparison quality:

• Product description
• Return policy
• Shipping details
• Aggregate rating

A product missing a price is not treated as “almost ready.” It is missing information an automated shopper needs before it can compare the item responsibly.

Evidence, not guesses

Every paid audit row can include the trimmed Product markup that was evaluated. The present, missing, and advisory arrays make the result easy to turn into a ticket or QA checklist.

If a page cannot be fetched, it is returned as unreachable, is not graded, and is not charged. A timeout or HTTP block describes the request outcome—not the quality of the merchant’s catalog.

AI crawler access

When checkCrawlerAccess is enabled, the Actor adds one free access row per store. It separates crawlers used for search or user-requested retrieval from crawlers associated with model training.

The output includes:

• allowed
• blockedSearch
• blockedTraining
• unknown
• robotsStatus

A missing robots.txt file and an unreachable one are not treated as the same condition. A 404 means no rules were published; a failed request remains unknown.

Input

Field	Description
startUrls	Product page URLs, or store homepages when discovery is enabled
discoverProducts	Discover product pages from robots.txt and XML sitemaps
maxProducts	Maximum number of product pages considered in the run
checkCrawlerAccess	Add the free per-store AI crawler access row
onlyProblems	Omit ready pages; omitted pages are not charged
crawlerIdentity	Optional custom User-Agent for your organization

Start with 10–25 pages. Review the results, then increase maxProducts for a full catalog run.

Output

The default dataset contains three row types in one comprehensive Audit results view. Use rowType to distinguish them:

Product page audits

{
  "rowType": "audit",
  "ok": true,
  "url": "https://shop.example/products/legacy-tee",
  "name": "Legacy Fitted T-Shirt",
  "score": 63,
  "verdict": "partial",
  "noMarkup": false,
  "present": ["name", "brand", "price", "priceCurrency"],
  "missing": ["sku_or_gtin", "availability"],
  "advisory": ["returnPolicy", "shippingDetails"],
  "source": "jsonld",
  "evidence": "{"@type":"Product","name":"Legacy Fitted T-Shirt"..."
}

AI crawler access

One free row per store with robotsStatus, allowed crawlers, blocked search crawlers, blocked training crawlers, and unknown results.

Unreachable pages

Free, ungraded rows with URL, HTTP status, error message, and timestamp.

Score interpretation

Score	Verdict	Meaning
90–100	ready	The page exposes the core product and offer signals
60–89	partial	The page is usable but has material gaps
0–59	not-ready	Required product or offer information is missing

Read noMarkup separately. “No Product node exists” and “the Product node is incomplete” usually require different fixes.

Pricing

Pay per event:

• $0.01 per Actor start at the default 512 MB memory
• $0.0025 per delivered product-page audit — $2.50 per 1,000
• AI crawler access rows are free
• Unreachable pages are free
• Pages omitted by onlyProblems are free
• Apify platform usage is included

Examples before any user-level pricing discounts:

Delivered audits	Estimated price
10	$0.035
100	$0.26
1,000	$2.51

The Actor respects the maximum total charge configured for the run and stops before continuing unpaid work.

Reliability and politeness

Requests to one host are spaced out, response sizes and timeouts are capped, and HTTP 429 receives one backed-off retry. The Actor reads public pages only and does not log in, solve challenges, bypass access controls, or submit purchases.

Limitations

• No browser rendering: product data available only after client-side JavaScript may be absent from the initial HTML.
• Scores are deterministic technical triage, not certification or a guarantee of placement in any AI answer.
• Crawler access is evaluated at the site root; path-specific rules may differ.
• Results are a point-in-time snapshot and can change when page markup or robots.txt changes.
• Product discovery depends on public robots.txt, sitemaps, and fallback homepage links. It cannot find intentionally hidden catalog pages.

Automation

Run the Actor on an Apify schedule for recurring catalog QA, call it through the Apify API, or connect the structured dataset to a spreadsheet, database, webhook, or downstream Actor.

Responsible use and support

Audit only public pages you are permitted to access, and follow applicable website terms, laws, and organizational policies. The output is technical information, not legal, regulatory, or commercial advice.

For support, open an Actor issue with a sample URL, the run ID, and the dataset row that looked unexpected. Never include credentials or private customer data.