Technical SEO Audit & Website Crawler - Core Web Vitals

Crawl any public website and turn technical SEO problems into structured, prioritized data. Every crawled page receives a 0-100 score, an A-F grade, severity counts, detailed checks, and actionable recommendations.

The Actor audits metadata, crawlability, indexability, heading structure, content quality signals, internal and external links, images, JSON-LD, social tags, security headers, duplicate metadata, robots.txt, sitemaps, and optional Google PageSpeed/Core Web Vitals.

Why use this SEO audit Actor?

• Report-ready issue objects - every problem has a code, severity, category, message, recommendation, and affected field.
• Site-wide crawling - follow internal links with depth, page, domain, subdomain, include, and exclude controls.
• Transparent scoring - critical, high, medium, and low issues feed a predictable 0-100 score.
• Duplicate detection - identify repeated titles and meta descriptions after the crawl completes.
• Optional broken-link checks - verify internal and external links with bounded extra requests.
• Optional Core Web Vitals - enrich selected pages with PageSpeed, Lighthouse, LCP, CLS, INP, TBT, and Speed Index.
• Billing-safe limits - the crawler respects both maxPages and the user's maximum-charge limit.
• Fast HTTP crawler - no browser is required for normal HTML audits.

Quick start

{
  "startUrls": ["https://example.com"],
  "maxPages": 25,
  "maxDepth": 3
}

For a focused content-section audit:

{
  "startUrls": ["https://example.com"],
  "maxPages": 100,
  "maxDepth": 5,
  "includePatterns": ["https://example.com/blog/*"],
  "excludePatterns": ["*/tag/*", "*/author/*", "*?preview=*"]
}

What the Actor checks

Metadata and indexability

• Missing, short, or long title tags
• Missing or poorly sized meta descriptions
• Missing canonical links
• noindex and restrictive robots directives
• Missing HTML language and mobile viewport
• HTTP error pages

Headings and content

• Missing or multiple H1 headings
• Skipped heading levels
• Visible word count and thin-content warnings
• Content-to-HTML ratio
• Duplicate titles and descriptions across pages

Links and images

• Internal, external, and unique link counts
• Optional broken and unreachable link checks
• Images missing alt text
• Images without width and height attributes
• Native lazy-loading counts

Structured data and social previews

• JSON-LD block count and discovered schema types
• Invalid JSON-LD
• Open Graph completeness
• Twitter/X card presence
• Hreflang count

Performance and security

• HTML response size
• HSTS and Content-Security-Policy presence
• Optional PageSpeed performance, SEO, accessibility, and best-practices scores
• Optional LCP, CLS, INP, Total Blocking Time, and Speed Index

Output

Each default dataset row represents one crawled page:

{
  "finalUrl": "https://example.com/pricing",
  "statusCode": 200,
  "score": 82,
  "grade": "B",
  "indexable": true,
  "title": "Pricing - Example",
  "titleLength": 17,
  "metaDescriptionLength": 148,
  "h1Count": 1,
  "wordCount": 642,
  "imagesMissingAlt": 2,
  "brokenLinksCount": 1,
  "criticalIssues": 0,
  "highIssues": 1,
  "issueCount": 6,
  "issues": [
    {
      "code": "short-title",
      "severity": "medium",
      "category": "metadata",
      "message": "Title is only 17 characters.",
      "recommendation": "Use a descriptive title that clearly communicates page intent.",
      "field": "title"
    }
  ]
}

The default key-value store also contains:

• SUMMARY - average score, grade, issue totals, indexability, failed pages, and top issue codes.
• ISSUES - one flat report-ready list of all issues across all pages.
• SITE_SIGNALS - robots.txt status, content, and discovered sitemap URLs.

PageSpeed and Core Web Vitals

Enable includePageSpeed to call Google PageSpeed Insights for up to pageSpeedMaxPages. A Google API key is optional but recommended for reliable quota. The key is treated as secret input and never appears in output.

PageSpeed calls are slower and subject to Google's quota, so use them on key templates rather than every URL in a large crawl.

Broken-link checking

Enable checkBrokenLinks to test discovered links after crawling. The Actor first sends a HEAD request and falls back to GET when a server rejects HEAD. Use maxLinksToCheck to bound runtime and third-party traffic.

SEO score

The score starts at 100 and deducts:

Severity	Deduction
Critical	20
High	8
Medium	4
Low	2

The score is a prioritization aid, not a promise of rankings. Search performance also depends on relevance, authority, content quality, competition, and off-page signals.

Use cases

• SEO agency prospect and client audits
• Site migration and redesign QA
• Scheduled technical SEO monitoring
• Programmatic site-quality dashboards
• Content inventory cleanup
• E-commerce category and product-page audits
• Pre-launch checks for marketing teams

FAQ

Does the Actor render JavaScript?

The default crawler audits server-delivered HTML. This makes it fast and inexpensive. If a site renders all SEO content only in the browser, server HTML may appear thin; such a site also deserves review because search crawlers need dependable renderable content.

Does this replace Google Search Console?

No. Search Console supplies Google's indexing and traffic data. This Actor supplies crawlable page-level diagnostics that can complement it.

Can I audit multiple sites in one run?

Yes. Add multiple startUrls; maxPages is shared across the entire run.

Will it crawl login, cart, or checkout pages?

Use excludePatterns to keep sensitive or low-value areas out of the crawl. Common exclusions are included in the default form.

Is a proxy required?

Usually not. Add an Apify proxy only when a public site rate-limits or blocks direct requests.