# AI Search Visibility Audit (`muhammadafzal/ai-search-visibility-audit`) Actor

Audit public website pages for AI search discoverability, crawler access, structured data, answer-ready content, trust signals, and citation readiness.

- **URL**: https://apify.com/muhammadafzal/ai-search-visibility-audit.md
- **Developed by:** [Muhammad Afzal](https://apify.com/muhammadafzal) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 audited pages

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## AI Search Visibility Audit

Audit a public website for the technical and content signals that help search engines and AI answer systems discover, understand, and cite its pages.

This Actor is built for SEO, AEO, GEO, content, and technical marketing teams. Give it one public website URL and it returns a scored record for every audited page plus an aggregate `OUTPUT` report with the most frequent issues and prioritized actions. It uses direct public HTTP access and deterministic rules, so buyers do not need an LLM key.

> Scope: this is a website readiness audit. It does not query or claim rankings, mentions, or citations inside ChatGPT, Gemini, Claude, Perplexity, or Google AI Overviews.

### What the audit measures

Every default-dataset item represents one HTML page and contains evidence rather than a generic pass/fail label.

| Field | What it tells you |
|---|---|
| `url`, `finalUrl`, `statusCode` | Requested page, redirect destination, and HTTP outcome |
| `score`, `grade` | Deterministic readiness score from 0–100 and grade A–F |
| `categoryScores` | Crawlability (20), content clarity (25), structured data (20), trust (20), and AI discovery (15) |
| `title`, `metaDescription`, `canonicalUrl`, `h1Count` | Core search and page-identity signals |
| `wordCount`, `answerBlockCount` | Substantive text and question-led concise answer sections |
| `structuredDataTypes` | Usable JSON-LD `@type` values, including nested `@graph` entries |
| `hasAuthor`, `publishedAt`, `modifiedAt` | Attribution and freshness evidence |
| `externalCitationCount` | Unique external domains linked as potential supporting sources |
| `aiCrawlerAccess` | Path-specific `robots.txt` access for OAI-SearchBot, ChatGPT-User, GPTBot, ClaudeBot, PerplexityBot, and Google-Extended |
| `llmsTxt`, `sitemap` | Observed discovery-resource availability |
| `issues`, `recommendations` | Evidence-backed gaps and actions for that page |

The run also writes `OUTPUT` to the default key-value store. It contains the average score, aggregate grade, site-level access signals, warning list, top issues, prioritized recommendations, and transparent event-cost estimate.

### When to use it

Use this Actor to:

- audit a site before or after an SEO/AEO redesign;
- find pages that lack canonical URLs, schema, authors, dates, citations, or concise answers;
- review how `robots.txt` treats common AI crawler user agents;
- turn a site-wide audit into JSON, CSV, Excel, an API response, or an automation input;
- create a repeatable baseline for scheduled technical checks.

Do not use it to measure actual brand visibility in answer engines, access authenticated content, bypass CAPTCHAs or paywalls, or make a legal promise that any optimization will cause an AI system to cite a page.

### Input

| Field | Type | Default | Meaning |
|---|---:|---:|---|
| `startUrl` | string | `https://www.apify.com/` | One absolute public HTTP(S) page or website URL |
| `maxPages` | integer | `10` | Maximum paid page records, 1–100 |
| `maxDepth` | integer | `2` | Internal-link traversal depth, 0–5 |
| `includeSubdomains` | boolean | `false` | Follow subdomains of the starting host |
| `respectRobotsTxt` | boolean | `true` | Skip paths disallowed to generic crawlers |
| `proxyConfiguration` | object | direct access | Optional Apify proxy configuration |

The Actor starts with the requested URL, checks `robots.txt`, `/llms.txt`, and declared or conventional XML sitemaps, then fills the page limit from sitemap candidates and in-scope links. It never follows unrelated external domains. Public URL validation rejects embedded credentials, localhost, private IP literals, and hostnames resolving to private addresses.

#### Console input example

```json
{
  "startUrl": "https://example.com/",
  "maxPages": 25,
  "maxDepth": 2,
  "includeSubdomains": false,
  "respectRobotsTxt": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

#### API example

Replace `YOUR_TOKEN` and the Actor identifier with values from your Apify account:

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/muhammadafzal~ai-search-visibility-audit/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrl":"https://example.com/","maxPages":10,"maxDepth":2}'
```

For production integrations, send the token in an authorization header rather than committing it to source code.

### Output example

```json
{
  "url": "https://example.com/guides/ai-search",
  "finalUrl": "https://example.com/guides/ai-search",
  "statusCode": 200,
  "title": "AI Search Optimization Guide",
  "metaDescription": "A practical guide to AI search optimization.",
  "canonicalUrl": "https://example.com/guides/ai-search",
  "h1Count": 1,
  "wordCount": 1260,
  "indexable": true,
  "structuredDataTypes": ["Article", "BreadcrumbList", "Organization"],
  "answerBlockCount": 3,
  "externalCitationCount": 7,
  "hasAuthor": true,
  "publishedAt": "2026-01-15",
  "modifiedAt": "2026-05-20",
  "aiCrawlerAccess": {
    "OAI-SearchBot": true,
    "ClaudeBot": true,
    "PerplexityBot": true
  },
  "llmsTxt": "present",
  "sitemap": "present",
  "score": 82,
  "grade": "B",
  "categoryScores": {
    "crawlability": 20,
    "contentClarity": 20,
    "structuredData": 15,
    "trust": 12,
    "aiDiscovery": 15
  },
  "issues": ["No question-led concise answer block was detected."],
  "recommendations": ["Add question headings followed by concise, self-contained answers where useful."],
  "auditedAt": "2026-08-31T12:00:00.000Z"
}
```

### Scoring method

The score is intentionally explainable. It awards fixed points for observed signals: successful and indexable canonical pages; useful titles, descriptions, headings, body text, and answer blocks; valid page-purpose and breadcrumb JSON-LD; author, date, identity, and source-link evidence; AI crawler access; `llms.txt`; and XML sitemap availability.

The score is a diagnostic prioritization tool, not a search-engine standard or ranking guarantee. Missing `llms.txt`, for example, is reported as an optional discovery opportunity rather than proof that a page cannot appear in AI search.

### Pay-per-event pricing

| Event | Price |
|---|---:|
| Audit start | $0.00005 per run |
| Audited page | $0.002 per dataset record |

A 10-page run costs up to **$0.02005** in Actor events. A 25-page run costs up to **$0.05005**. Pages blocked by robots, access challenges, non-HTML responses, and failed requests do not create dataset-item events. The Actor reduces its page limit when the run's maximum charge cannot cover all requested page records.

### Reliability and limits

- Direct static HTML is the primary access path. Optional Apify proxy settings are supported for ordinary IP-based access problems.
- Requests use bounded timeouts, two retries for transient statuses, a finite request multiplier, and a maximum of 100 delivered pages.
- HTTP 401, 403, 429, challenge pages, invalid targets, and truthful empty crawls are distinguished in `OUTPUT`.
- Valid partial results are preserved when later pages fail.
- JavaScript-only content that is absent from returned HTML cannot be audited by this version.
- The Actor does not solve CAPTCHAs, use stolen sessions, bypass authentication, or crawl private networks.

### Responsible use

Audit websites you are allowed to access. Respect applicable terms, robots directives, copyright, privacy, and data-protection requirements. The Actor extracts public technical and content signals; review outputs before using them for consequential decisions. Search and AI crawler behavior changes independently of this Actor, so validate critical decisions with the relevant platform's current documentation.

For support, include the run ID, start URL with sensitive query parameters removed, expected outcome, and the relevant `OUTPUT` warnings.

# Actor input Schema

## `startUrl` (type: `string`):

Use this to choose the public website to audit. Enter one absolute HTTP(S) URL, for example https://www.apify.com/. The Actor does not access logins, intranets, or private network addresses.

## `maxPages` (type: `integer`):

Use this to cap dataset records and event cost. The Actor audits 1–100 HTML pages; default 10. This is not a request or link discovery limit.

## `maxDepth` (type: `integer`):

Use this to control internal-link traversal from the start URL. Choose 0–5; default 2. Depth 0 audits only the start page plus sitemap candidates needed to fill the page limit.

## `includeSubdomains` (type: `boolean`):

Use this when the audit should follow links on subdomains of the start domain. Default false. Unrelated domains and external citation links are never crawled.

## `respectRobotsTxt` (type: `boolean`):

Use this to skip pages disallowed to generic crawlers. Default true and recommended. AI-bot directives are always measured as audit signals; disabling this does not bypass access controls.

## `proxyConfiguration` (type: `object`):

Use this only when ordinary public HTTP access needs an Apify proxy. The default is a direct request. This does not defeat authentication, CAPTCHAs, paywalls, or other access controls.

## Actor input object example

```json
{
  "startUrl": "https://www.apify.com/",
  "maxPages": 10,
  "maxDepth": 2,
  "includeSubdomains": false,
  "respectRobotsTxt": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

Schema-validated page-level audit records.

## `summary` (type: `string`):

Aggregate score, crawler access evidence, top issues, and prioritized recommendations.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrl": "https://www.apify.com/"
};

// Run the Actor and wait for it to finish
const run = await client.actor("muhammadafzal/ai-search-visibility-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrl": "https://www.apify.com/" }

# Run the Actor and wait for it to finish
run = client.actor("muhammadafzal/ai-search-visibility-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrl": "https://www.apify.com/"
}' |
apify call muhammadafzal/ai-search-visibility-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,muhammadafzal/ai-search-visibility-audit"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/XbyuUDEZC3HcpiMxZ/builds/lgW7VHEodyMhQ2T7v/openapi.json