SEO and AI Search Site Audit: scores, ranked fixes and a shareable report

Find the technical issues that stop a website’s pages from being read, indexed or reused in AI search. Each HTML page starts with a one-line verdict, followed by its 0–100 scores, evidence and specific fixes. The site report combines those findings into a ranked work list.

$0.01 per successfully audited page, the same price on every Apify plan. The HTML report, site summaries and optional change comparison are included. Failed pages produce a note without a page charge. No browser or AI key is needed. Start with one URL; the default page limit is 100 per site.

What you get

Field	What it holds
Page verdict and scores	A plain verdict plus technical, on-page, content, structured-data, AI-search-readiness and overall scores. Critical search exclusions cannot receive a green overall score.
Indexability evidence	HTTP status, redirect chain, meta robots, scoped X-Robots-Tag, canonical targets and the matching robots.txt rule. Actual indexing is not verified.
Page content and metadata	Titles and descriptions with lengths and site duplicates; H1/H2 outline, main-text word count, script-shell evidence, image alt coverage and sharing tags.
Links and language variants	Internal incoming and outgoing links, click depth, known broken targets and hreflang return-link checks where both pages were fetched. Optional bounded external checks.
Structured data	JSON-LD and microdata types, JSON parse errors, selected required and recommended property checks for Organization, Product, Article, FAQPage, LocalBusiness and BreadcrumbList.
AI access declarations	Search, training and user-requested crawler policies kept distinct; Content-Signal lines, scoped page directives, optional llms.txt format checks and static-text evidence.
Site report	REPORT.html is a standalone web page with scores, ranked fixes, affected URLs, duplicate and near-duplicate clusters, orphan candidates, sitemap coverage, redirects and HTTP errors. SUMMARY contains the same site analysis as JSON.
Change tracking	Use a named key-value store to compare pages, issues and scores between comparable runs. Partial runs do not establish that an unseen page was removed.

What people use it for

• Agency SEO and GEO audits: Turn a crawl into a prioritised implementation list with evidence, URL lists and a report that can be shared.
• Release checks: Find accidental noindex directives, broken internal links, changed canonicals or pages that became script shells.
• AI crawler policy review: See whether search access is disallowed while preserving a separate view of training opt-outs.
• Ongoing monitoring: Schedule a stable crawl scope and review added or resolved issues against the previous baseline.

How to use it

1. Enter one or more public Start URLs. URLs on the same host share a page budget; the www and non-www forms belong to the same site.
2. Choose the maximum pages per site and optional include/exclude glob patterns. Keep the request rate at or below two per second.
3. Set Maximum cost per run in Apify. For a first check, use a small page limit.
4. Open the page dataset for individual verdicts, or REPORT.html for the site overview and ranked fixes.

Input

Field	Example	Notes
startUrls	["https://example.com/"]	Public URLs. Bare domains are accepted. Empty input uses the small example site.
maxPagesPerSite	100	1–10,000 candidate pages per site, including failed candidates. Pages are read at up to 2 a second, so 1,000 pages take about 9 minutes; for more than about 5,000 pages, raise the run timeout (Run options) above the default hour.
includePatterns	["/guides/"]	Optional glob patterns on the path or full URL. The seeds must match too.
excludePatterns	["/cart", "/search?"]	Exclusions override inclusions.
checkExternalLinks	false	Optional supplementary HEAD checks, with a GET fallback for servers rejecting HEAD.
compareWithPrevious	false	Enable comparison with a previous comparable audit.
comparisonStoreName	"site-audit-history"	Required with comparison. Use separate names for independent monitoring projects.
requestsPerSecond	2	0.1–2 requests per second per site; longer robots Crawl-delay values are respected.
maxRequestsPerSite	Automatic	HTTP budget including auxiliary files, redirects and retries. Set a smaller cap for sensitive sites.
maxSitemaps	12	Read up to this many XML sitemap or index files, including gzip. Zero disables discovery.
maxExternalLinksPerSite	50	Maximum external targets. External work also has a 10–60-second site budget and five-second request timeouts.
requestTimeoutSeconds	30	5–60 seconds for page requests, with 16 MB compressed and 32 MB decoded body limits.
maxSiteSeconds	1800	Per-site time budget, up to 43,200 seconds. Raise it for large or slow sites.

Output

Each result is one row in the run's dataset (download it as JSON, CSV, Excel, XML or HTML, or read it through the API). An example row (values shortened):

{
  "verdict": "1 critical: No useful text without JavaScript",
  "url": "https://example.com/",
  "overallScore": 49,
  "httpStatus": 200,
  "indexability": {
    "state": "indexable"
  },
  "wordCount": 0,
  "staticContent": {
    "mainTextPresent": false,
    "mainWordCount": 0,
    "bodyWordCount": 1,
    "scriptCount": 6,
    "status": "script-shell"
  },
  "scores": {
    "technical": 100,
    "onPage": 100,
    "content": 20,
    "structuredData": 100,
    "aiSearchReadiness": 20
  },
  "issues": [
    {
      "code": "NO_STATIC_TEXT",
      "severity": "critical",
      "category": "aiSearchReadiness",
      "message": "No useful text without JavaScript",
      "howToFix": "Serve the main text in the initial HTML; inspect whether scripts are required."
    }
  ],
  "charged": true
}

This shortened example comes from a real public-page test on 25 October 2026. The site address was replaced; no source marketing text is reproduced. The numeric values and observed issue are from that run. SUMMARY contains per-site scores, the formula, coverage limits, route escalations and event counts. REPORT.html contains the shareable report.

Pricing

• $0.01 per audited HTML page, the same price on every Apify plan.
• Only successful HTML audits generate a page event. HTTP errors, clear soft 404s, refusals, robots exclusions, invalid inputs, non-HTML files and duplicate final destinations are free notes.
• Site summaries, the HTML report and optional comparison have no separate report event. There is no model charge.
• Apify’s $0.00005 start charge counts towards the buyer’s Maximum cost per run. Set at least $0.01005 to allow one page event.

What typical jobs cost at the list price:

Job	Cost
1 audited page	$0.01005 including the start charge
25 audited pages	$0.25005 including the start charge
100 audited pages	$1.00005 including the start charge
1,000 audited pages	$10.00005 including the start charge

The same price applies on every Apify plan. Maximum cost per run limits page events; the Actor checks room before fetching more pages and still writes free notes and its report when the limit stops discovery. A restart uses stable result IDs to avoid charging delivered pages twice.

Tips

• Use the same start URLs, patterns and limits for scheduled comparisons. Run the schedule sequentially; named stores have a last-writer-wins baseline.
• Start at the home page for meaningful click depth. Sitemap-only pages have unknown depth until the observed link graph connects them.
• A zero inbound-link count in a partial crawl is an orphan candidate, not proof that no page on the whole site links there.
• Read the matching rule and intended purpose before changing an AI policy. A training opt-out is not automatically an AI-search problem.
• The strongest individual signals are explicit noindex, an unintended search crawl block, broken targets and missing static main text. Advisory text lengths are not search-engine limits.

Use it through the API

Python

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("zaiq/seo-site-audit").call(run_input={"startUrls": ["https://example.com/"], "maxPagesPerSite": 10})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row)

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('zaiq/seo-site-audit').call({"startUrls": ["https://example.com/"], "maxPagesPerSite": 10});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

HTTP (one call, results back)

curl -X POST 'https://api.apify.com/v2/acts/zaiq~seo-site-audit/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{"startUrls": ["https://example.com/"], "maxPagesPerSite": 10}'

Integrations and schedules

• Schedules: run it every hour, day or week from the Apify Console (Schedules) and keep a history of results.
• Make, Zapier, n8n, Google Sheets, Slack, Airtable: connect it through Apify's integrations, or send each finished run to a webhook.
• Datasets: results stay in the run's dataset; export them or read them by API whenever you need them.

Use it with AI agents (MCP)

Add Apify's MCP server (https://mcp.apify.com) to Claude, ChatGPT, Cursor, VS Code or any MCP client. The agent can find this Actor and run it for you: ask it for "audit this public website for technical SEO and AI search access, then rank the fixes". AI agents can also pay for runs directly through Apify's agentic payments.

Limits

• This is a static HTTP audit. It does not run JavaScript, render layout, download page assets or measure Lighthouse, Core Web Vitals, rankings, backlinks or real AI citations. Page weight is HTML transfer size only.
• The crawler obeys robots.txt, uses an ordinary browser User-Agent and never sends another company’s crawler identity. Public data only: no logins, CAPTCHA solving or anti-bot tokens. A 429 stops further requests to the affected site for that run.
• Cloud requests may try a direct connection, a datacenter route and fresh residential sessions when an address is refused. Every escalation is recorded. Genuine script shells are findings, not a reason to retry on another address.
• Robots access is a declared policy for the paths checked. Bot IP restrictions and actual vendor behaviour are untested. User-triggered fetchers can treat robots differently. Bytespider’s current vendor documentation could not be confirmed; the requested token is still checked.
• Structured-data checks cover selected search-feature profiles and FAQ structural completeness, not every schema rule. FAQ checks do not imply eligibility for a Google rich result. Article and Organization have no universal required properties. Recommendations are advisory; eligibility for rich results is not guaranteed.
• llms.txt is an optional community proposal. Only the site-root file receives a basic v2 structure and link-format check. Presence or absence does not change the score and cannot guarantee citations.
• Site scores average the observed pages. Each category starts at 100 and deducts 30/15/7/3/0 points for critical/high/medium/low/info issues, once per code. Overall weights are technical 30%, on-page 20%, content 20%, structured data 10%, AI readiness 20%. No static text caps content and AI readiness at 20; an AI-search disallow caps AI readiness at 50. Noindex, a search crawl block or no static text caps page overall at 49. Site overall cannot exceed the mean capped page overall. The technical site score also includes 20% observed HTTP availability. The complete formula is in every report.
• Missing llms.txt, training opt-outs, text-length advice and unverified targets do not lose points. No usable HTML means no score.
• Sitemap inventory, link discovery and page budgets are bounded and may be partial. A page retains up to 2,000 internal and 1,000 external destinations. Near-duplicate clusters are review candidates based on a bounded five-word shingle sketch.
• One successful public crawl was tested at 1,000 pages. The 10,000-page input range also passed an offline capacity check; cloud memory, larger live crawls and proxy behaviour depend on the source.

FAQ

Does this need an AI API key?

No. The checks and scores are deterministic and based on the site’s responses.

Does a high score guarantee indexing or AI citations?

No. It describes the checks completed on observed pages, with the formula and coverage shown.

Is blocking model training an error?

No. Training controls are reported separately and do not reduce the score. Search-specific restrictions are flagged for review.

Are failed pages billed?

They do not generate a page event. They remain useful free notes in the dataset. The run’s start charge is separate.

Will every outgoing link be checked?

No. Internal targets are checked when reached by the crawl. External checks are optional, sampled and bounded by count and time. Unchecked targets remain unknown.

How do comparisons work?

A named store holds a baseline for each site and crawl scope. Matched successful pages can show resolved issues. An incomplete or all-failed run cannot erase the last better baseline.

Can I use a subdirectory?

Yes. Add include patterns such as /guides/ if the audit must stay in that directory. The default scope is the host, not the seed path.

What does the HTML report contain?

Site scores, the exact formula, ranked fixes with URLs, coverage, duplicates, orphan candidates, crawler declarations and page evidence. It opens as a standalone file.

Related Actors from Zaiq

• ai-visibility-audit: measure how AI answers mention and cite a brand
• ai-answer-tracker: track AI answers over time
• llm-website-crawler: extract website content as Markdown for retrieval workflows

Support

Questions, a site that changed, or a field you need? Open an issue on this Actor's Issues tab: we answer there, usually within a day.

---

Also found as: SEO site audit, technical SEO crawler, AI search readiness, GEO audit, robots.txt checker, structured data audit, hreflang checker, SEO audit report, website indexability.