Website SEO, Accessibility & AI Readiness Auditor

Audit public websites for technical SEO, accessibility signals, security headers, broken links, lightweight performance risks, structured data, robots.txt, sitemap.xml, llms.txt, and major AI crawler directives. Get deterministic 0–100 scores, concrete evidence, prioritized fixes, structured dataset views, and a client-ready HTML report. 🔎

This Actor is built for SEO agencies, web developers, accessibility specialists, content teams, and website owners who need more than a raw crawl. It converts page-level signals into an actionable audit without a paid API and without calling another Actor.

Quick Start 🚀

1. Add one or more website URLs or domains.
2. Keep Maximum pages per site at 10 for the first run.
3. Leave every audit module enabled.
4. Click Save & Start.
5. Open Audit overview for the prioritized page list or Client-ready HTML report for the visual summary.

Bare domains are accepted, so example.com becomes https://example.com/. The Actor follows same-site HTML links, prioritizes sitemap URLs, respects robots.txt by default, and stops at the configured per-site page limit.

What It Audits

Area	Checks
Technical SEO	HTTP status, indexability, robots directives, canonical tags, page title, meta description, H1 structure, structured data, internal/external links
Crawl discovery	robots.txt, declared and default sitemaps, sitemap indexes, same-site link discovery
Broken links	Unique internal and external destinations with status/error evidence mapped back to affected pages
Accessibility signals	HTML language, image alt attributes, programmatic form labels, accessible link/button names, main landmarks
AI readiness	llms.txt, JSON-LD context, semantic main content, and access rules for GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-SearchBot, Google-Extended, PerplexityBot, CCBot, and Bytespider
Security	HTTPS, HSTS, Content Security Policy, nosniff, Referrer-Policy, Permissions-Policy, and frame embedding protection
Performance hints	Initial HTML response time, transfer size, script pressure, render-blocking script hints, stylesheets, iframes, DOM size, and image lazy-loading coverage
Conversion structure	Forms, phone/email links, and common call-to-action labels
Mobile signals	Responsive viewport declaration, responsive image markup, and legacy layout-table hints

Why Use This Website Audit Actor? ✨

• Evidence instead of vague grades: every deduction has an issue code, severity, observed value, and concrete remediation action.
• Agency-ready deliverables: send the HTML report to a client or import flat CSV rows into a spreadsheet.
• AI search visibility checks: review machine-readable context and crawler policy alongside conventional SEO.
• Predictable cost: one visible dataset result per audited URL; no hidden API or third-party Actor charge.
• Automation friendly: purpose-built dataset views and JSON output work with the Apify API, webhooks, Make, Zapier, and AI agents.
• Low infrastructure overhead: the Actor performs direct HTTP analysis without launching a browser.

URL-Level Output

Every audited page creates one dataset row with:

• pageScore and siteScore from 0 to 100
• category scores for SEO, accessibility, AI readiness, security, and performance hints
• critical, warning, informational, and total issue counts
• indexability and robots evidence
• title, description, H1, canonical, and structured data details
• checked and broken link evidence
• image, form, CTA, mobile, and accessibility signals
• security header and performance evidence
• llms.txt and AI crawler access status
• complete issues, evidence, and recommendations objects

See examples/output.json for a representative row.

Structured Dataset Views 📊

The default dataset has seven views:

• Audit overview: page/site scores, issue counts, indexability, broken links, and first actions.
• SEO & indexability: metadata, canonicals, robots, structured data, and link signals.
• Accessibility signals: fast HTML triage before manual WCAG testing.
• AI readiness: llms.txt, schema context, semantic structure, and AI crawler directives.
• Security & performance: headers, HTTPS, mobile, timing, transfer, script, and DOM hints.
• Evidence & recommendations: complete issue and remediation objects.
• All URL-level data: the full structured contract for APIs and integrations.

Key-Value Store Reports

Each run stores:

• REPORT.html: client-ready visual report
• REPORT.json: run totals, site summaries, page rows, and request diagnostics
• SUMMARY.json: compact run and site summary
• OUTPUT.json: URL-level result export
• OUTPUT.csv: spreadsheet-ready flat export
• ERRORS.json: transparent request and sitemap diagnostics

Example Input

{
  "startUrls": [
    { "url": "https://example.com" },
    { "url": "https://www.apify.com" }
  ],
  "maxPagesPerSite": 10,
  "includeLinkChecks": true,
  "includeAiReadiness": true,
  "includeAccessibility": true,
  "includePerformanceHints": true,
  "respectRobotsTxt": true,
  "maxLinkChecksPerSite": 100,
  "maxSitesConcurrency": 2,
  "linkCheckConcurrency": 8,
  "requestTimeoutSecs": 25,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Input Reference ⚙️

• startUrls: 1–20 public URLs or domains. The request-list UI accepts one URL per row.
• maxPagesPerSite: 1–200 same-site HTML pages per website.
• includeLinkChecks: check unique internal and external link destinations.
• includeAiReadiness: inspect llms.txt, AI crawler policy, structured context, and semantic content.
• includeAccessibility: run deterministic accessibility markup checks.
• includePerformanceHints: collect lightweight response and document complexity signals.
• respectRobotsTxt: avoid following disallowed discovered URLs. The submitted start URL is still fetched once.
• maxLinkChecksPerSite: 0–1,000 unique link checks per website.
• maxSitesConcurrency: audit 1–8 input websites in parallel.
• linkCheckConcurrency: perform 1–20 link status checks in parallel per website.
• requestTimeoutSecs: 5–90 seconds per request.
• proxyConfiguration: optional Apify Proxy settings.

How Scoring Works

The score is deterministic and explainable:

• SEO: 40% of the default score
• Accessibility: 20%
• AI readiness: 15%
• Security: 15%
• Performance hints: 10%

Disabled optional modules are removed from the weighted calculation rather than treated as perfect. Each finding has an explicit deduction; category scores cannot fall below zero. Page grades are:

Score	Grade
90–100	Excellent
75–89	Good
60–74	Needs work
40–59	Poor
0–39	Critical

The site score is the average of the audited page scores. It describes only the sampled URLs and is not a search-ranking guarantee.

Recommended Audit Sizes

Use case	Pages per site	Link checks
Fast sales qualification	3–5	25
Standard agency audit	10–25	100
Content-heavy website	50–100	250
Large technical crawl sample	200	500–1,000

Start small. Sitemap and same-site discovery make the first 10–25 pages useful for prioritization, but a capped sample cannot prove that every website URL is healthy.

Run Locally

Requirements: Node.js 22 or newer.

npm install
export APIFY_LOCAL_STORAGE_DIR=./storage
printf '%s\n' '{"startUrls":[{"url":"https://example.com"}],"maxPagesPerSite":2}' > storage/key_value_stores/default/INPUT.json
npm start

Run the network-free unit tests:

$npm test

Run a real public-web smoke test:

$npm run smoke -- https://example.com

The smoke script uses the same HTTP client, sitemap discovery, page auditor, scorer, site aggregation, and HTML renderer as the Actor.

Run on Apify

npx --yes apify-cli login
npx --yes apify-cli push

Then open the Actor in Apify Console, paste the example input, and verify the dataset views plus REPORT.html. See APIFY_PUBLISHING_CHECKLIST.md for the exact publication process.

Scheduling and Agency Workflows

• Run monthly technical health checks and send a webhook when critical issue counts increase.
• Audit prospect websites in small batches and prioritize outreach by siteScore.
• Export OUTPUT.csv into an agency audit workbook.
• Send REPORT.html to an account manager for review before a client meeting.
• Compare stored REPORT.json snapshots in your own automation to identify regressions.

This version is a point-in-time auditor, not a stateful change monitor. Historical comparisons can be built with scheduled runs and retained datasets.

Important Limitations

• The Actor analyzes server-returned HTML and does not render client-side JavaScript. JavaScript-only metadata or links may not appear.
• Performance results are lightweight technical hints, not Lighthouse, CrUX, Core Web Vitals, or laboratory browser measurements.
• Accessibility checks are deterministic markup triage, not a WCAG conformance statement. Manual keyboard, screen-reader, contrast, and interaction testing remains necessary.
• Security header checks are not a penetration test or vulnerability assessment.
• llms.txt and AI crawler behavior are evolving. A discovered file or allowed crawler does not guarantee inclusion in an AI answer.
• Robots and noindex directives may be intentional. Confirm business and legal policy before changing them.
• External sites can block, throttle, redirect, or vary responses by region. Use Apify Proxy only when needed.

Responsible Use

Audit only public websites you are permitted to access. Respect target terms, robots policies, reasonable request rates, privacy obligations, and applicable law. Verify findings before changing a production website or contacting a prospect.

Pricing

Recommended Store pricing is $0.010 per URL-level result plus Apify's standard low synthetic Actor start event. A standard 10-page website audit therefore costs about $0.10, before any optional platform-usage pass-through. See PRICING.md for margin assumptions and setup details.