AI Search Visibility Audit

Audit a public website for the technical and content signals that help search engines and AI answer systems discover, understand, and cite its pages.

This Actor is built for SEO, AEO, GEO, content, and technical marketing teams. Give it one public website URL and it returns a scored record for every audited page plus an aggregate OUTPUT report with the most frequent issues and prioritized actions. It uses direct public HTTP access and deterministic rules, so buyers do not need an LLM key.

Scope: this is a website readiness audit. It does not query or claim rankings, mentions, or citations inside ChatGPT, Gemini, Claude, Perplexity, or Google AI Overviews.

What the audit measures

Every default-dataset item represents one HTML page and contains evidence rather than a generic pass/fail label.

Field	What it tells you
url, finalUrl, statusCode	Requested page, redirect destination, and HTTP outcome
score, grade	Deterministic readiness score from 0–100 and grade A–F
categoryScores	Crawlability (20), content clarity (25), structured data (20), trust (20), and AI discovery (15)
title, metaDescription, canonicalUrl, h1Count	Core search and page-identity signals
wordCount, answerBlockCount	Substantive text and question-led concise answer sections
structuredDataTypes	Usable JSON-LD @type values, including nested @graph entries
hasAuthor, publishedAt, modifiedAt	Attribution and freshness evidence
externalCitationCount	Unique external domains linked as potential supporting sources
aiCrawlerAccess	Path-specific robots.txt access for OAI-SearchBot, ChatGPT-User, GPTBot, ClaudeBot, PerplexityBot, and Google-Extended
llmsTxt, sitemap	Observed discovery-resource availability
issues, recommendations	Evidence-backed gaps and actions for that page

The run also writes OUTPUT to the default key-value store. It contains the average score, aggregate grade, site-level access signals, warning list, top issues, prioritized recommendations, and transparent event-cost estimate.

When to use it

Use this Actor to:

• audit a site before or after an SEO/AEO redesign;
• find pages that lack canonical URLs, schema, authors, dates, citations, or concise answers;
• review how robots.txt treats common AI crawler user agents;
• turn a site-wide audit into JSON, CSV, Excel, an API response, or an automation input;
• create a repeatable baseline for scheduled technical checks.

Do not use it to measure actual brand visibility in answer engines, access authenticated content, bypass CAPTCHAs or paywalls, or make a legal promise that any optimization will cause an AI system to cite a page.

Input

Field	Type	Default	Meaning
startUrl	string	https://www.apify.com/	One absolute public HTTP(S) page or website URL
maxPages	integer	10	Maximum paid page records, 1–100
maxDepth	integer	2	Internal-link traversal depth, 0–5
includeSubdomains	boolean	false	Follow subdomains of the starting host
respectRobotsTxt	boolean	true	Skip paths disallowed to generic crawlers
proxyConfiguration	object	direct access	Optional Apify proxy configuration

The Actor starts with the requested URL, checks robots.txt, /llms.txt, and declared or conventional XML sitemaps, then fills the page limit from sitemap candidates and in-scope links. It never follows unrelated external domains. Public URL validation rejects embedded credentials, localhost, private IP literals, and hostnames resolving to private addresses.

Console input example

{
  "startUrl": "https://example.com/",
  "maxPages": 25,
  "maxDepth": 2,
  "includeSubdomains": false,
  "respectRobotsTxt": true,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

API example

Replace YOUR_TOKEN and the Actor identifier with values from your Apify account:

curl -X POST \
  "https://api.apify.com/v2/acts/muhammadafzal~ai-search-visibility-audit/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrl":"https://example.com/","maxPages":10,"maxDepth":2}'

For production integrations, send the token in an authorization header rather than committing it to source code.

Output example

{
  "url": "https://example.com/guides/ai-search",
  "finalUrl": "https://example.com/guides/ai-search",
  "statusCode": 200,
  "title": "AI Search Optimization Guide",
  "metaDescription": "A practical guide to AI search optimization.",
  "canonicalUrl": "https://example.com/guides/ai-search",
  "h1Count": 1,
  "wordCount": 1260,
  "indexable": true,
  "structuredDataTypes": ["Article", "BreadcrumbList", "Organization"],
  "answerBlockCount": 3,
  "externalCitationCount": 7,
  "hasAuthor": true,
  "publishedAt": "2026-01-15",
  "modifiedAt": "2026-05-20",
  "aiCrawlerAccess": {
    "OAI-SearchBot": true,
    "ClaudeBot": true,
    "PerplexityBot": true
  },
  "llmsTxt": "present",
  "sitemap": "present",
  "score": 82,
  "grade": "B",
  "categoryScores": {
    "crawlability": 20,
    "contentClarity": 20,
    "structuredData": 15,
    "trust": 12,
    "aiDiscovery": 15
  },
  "issues": ["No question-led concise answer block was detected."],
  "recommendations": ["Add question headings followed by concise, self-contained answers where useful."],
  "auditedAt": "2026-08-31T12:00:00.000Z"
}

Scoring method

The score is intentionally explainable. It awards fixed points for observed signals: successful and indexable canonical pages; useful titles, descriptions, headings, body text, and answer blocks; valid page-purpose and breadcrumb JSON-LD; author, date, identity, and source-link evidence; AI crawler access; llms.txt; and XML sitemap availability.

The score is a diagnostic prioritization tool, not a search-engine standard or ranking guarantee. Missing llms.txt, for example, is reported as an optional discovery opportunity rather than proof that a page cannot appear in AI search.

Pay-per-event pricing

Event	Price
Audit start	$0.00005 per run
Audited page	$0.002 per dataset record

A 10-page run costs up to $0.02005 in Actor events. A 25-page run costs up to $0.05005. Pages blocked by robots, access challenges, non-HTML responses, and failed requests do not create dataset-item events. The Actor reduces its page limit when the run's maximum charge cannot cover all requested page records.

Reliability and limits

• Direct static HTML is the primary access path. Optional Apify proxy settings are supported for ordinary IP-based access problems.
• Requests use bounded timeouts, two retries for transient statuses, a finite request multiplier, and a maximum of 100 delivered pages.
• HTTP 401, 403, 429, challenge pages, invalid targets, and truthful empty crawls are distinguished in OUTPUT.
• Valid partial results are preserved when later pages fail.
• JavaScript-only content that is absent from returned HTML cannot be audited by this version.
• The Actor does not solve CAPTCHAs, use stolen sessions, bypass authentication, or crawl private networks.

Responsible use

Audit websites you are allowed to access. Respect applicable terms, robots directives, copyright, privacy, and data-protection requirements. The Actor extracts public technical and content signals; review outputs before using them for consequential decisions. Search and AI crawler behavior changes independently of this Actor, so validate critical decisions with the relevant platform's current documentation.

For support, include the run ID, start URL with sensitive query parameters removed, expected outcome, and the relevant OUTPUT warnings.