Website SEO Audit Crawler

Crawl a whole website and get a page-by-page technical SEO audit plus a site-wide summary: duplicate titles, broken internal links, missing meta descriptions, noindex pages, canonical problems and 20+ other checks. It uses plain HTTP requests (no browser), respects robots.txt, and is priced per audited page, so small sites cost cents.

Free-form bulk SEO audit of a whole website

Give the actor your homepage (or any page) and it follows links on the same host, auditing each page it reaches. Every page becomes one dataset row with the raw measurements (title, meta description, H1s, canonical, word count, link counts and so on) and an issues array of codes you can filter on. A SUMMARY record in the key-value store rolls the whole crawl up into issue counts, duplicates and broken links.

What it checks on every page

Area	Checks
Indexing	HTTP status, redirects, meta robots / X-Robots-Tag noindex, canonical missing / pointing elsewhere / invalid
Titles & meta	missing, too long (>60), too short (<20); meta description missing / >160 / <50; duplicates across the site
Content	H1 missing / multiple, thin content (<200 words), html lang, viewport meta
Links	internal, external and nofollow counts; broken internal links with the pages that link to them
Media	images missing alt
Social & schema	Open Graph tags, JSON-LD types (and invalid JSON-LD), hreflang alternates
Security & speed	mixed content on HTTPS, slow server response, large HTML

Find duplicate titles and broken internal links

The SUMMARY key-value record lists issue counts sorted by frequency, duplicate titles, duplicate meta descriptions, broken internal links (each with up to 20 referring pages) and average response time. That makes it a quick way to find broken links and duplicate metadata without opening every row.

Use cases

• Pre-launch or post-migration check: confirm redirects, canonicals and noindex tags are what you intended.
• Agency and freelancer audits: export one row per page to CSV or Excel and hand over a fix list.
• Scheduled monitoring: run weekly with an Apify schedule and compare issue counts over time.
• AI agents: flat input and an issues array per page suit LLM tools calling the actor through the Apify MCP server.

Input

{
  "startUrls": [{ "url": "https://example.com" }],
  "maxPages": 200,
  "maxDepth": 10,
  "includeSubdomains": false,
  "maxConcurrency": 10
}

• Start URLs (required): links on the same host are followed.
• Max pages (default 100): hard cap on pages audited, in total. With several sites in one run the budget is split evenly between them (e.g. 5 sites, 100 pages → 20 each), and pages a small site doesn't use are re-dealt to the larger ones.
• Max link depth (default 10): clicks away from the start URL.
• Include subdomains (default off): also crawl e.g. blog.example.com. example.com and www.example.com always count as the same site, so a start URL that redirects between them is crawled normally.
• Max concurrency (default 10, max 50): lower it for small servers.

Output

One dataset row per page. Example (abridged to a few real fields):

{
  "url": "https://example.com/pricing",
  "statusCode": 200,
  "responseTimeMs": 184,
  "title": "Pricing | Example",
  "titleLength": 17,
  "metaDescriptionLength": 0,
  "h1": ["Simple pricing"],
  "canonical": "https://example.com/pricing",
  "wordCount": 412,
  "imagesMissingAlt": 3,
  "jsonLdTypes": ["Organization"],
  "depth": 1,
  "issues": ["title-too-short", "missing-meta-description", "images-missing-alt", "missing-open-graph"]
}

Other fields available per row: finalUrl, htmlBytes, metaDescription, metaRobots, xRobotsTag, lang, hasViewport, images, internalLinks, externalLinks, nofollowLinks, openGraph, hreflang, mixedContentResources, error. Issue codes include missing-title, title-too-long, missing-h1, multiple-h1, noindex, redirected, invalid-canonical, canonicalized-elsewhere, thin-content, slow-response, mixed-content and fetch-failed.

Pricing

Pay per event: $0.005 per audited page ($5 per 1,000 pages), no subscription.

• 50-page site: about $0.25
• 200-page site: about $1.00
• 2,000-page site: about $10.00

Set Max pages (or a max cost per run) to cap spend; the actor stops when the charge limit is reached. Platform usage is billed by Apify according to your plan.

Limitations

• JavaScript is not executed: the HTML as served is audited, so client-rendered content and links may be missed.
• It reports on-page and technical signals; it does not measure rankings, backlinks or Core Web Vitals.
• Response time is measured from the crawler's location and is a rough indicator only.
• Audit only sites you own or are permitted to crawl.

FAQ

How do I find broken links on my website?

Run the actor from your homepage and open the SUMMARY record: broken internal links are listed with the pages that link to them.

How do I find duplicate title tags across a site?

The SUMMARY record includes duplicate titles and duplicate meta descriptions; each page row also has title and titleLength.

Can I audit a site that uses noindex or canonical tags?

Yes. Pages are still fetched and flagged with noindex, missing-canonical, invalid-canonical or canonicalized-elsewhere as appropriate.

Does it respect robots.txt?

Yes, the crawler honours robots.txt.

How much does a full website SEO audit cost?

$0.005 per page, so roughly $1 for 200 pages.

Can I crawl subdomains too?

Yes, switch on Include subdomains; by default only the start host is crawled.

Built and maintained with AI assistance. Issues and feature requests: use the Issues tab.

Related tools

Other actors by the same developer (same flat pay-per-result pricing, no subscription):

• App Store Reviews Scraper – Apple iOS, Multi-Country
• Article Scraper & Text Extractor – Clean Markdown for LLM/RAG
• Job Postings & Career Page Scraper – Workday, Greenhouse, Lever
• Bluesky Posts Scraper – Profiles & Threads, No Login
• Company Enrichment from Domain (Logo, Socials, Tech)
• DNS Lookup & WHOIS Domain Checker — SPF/DMARC, SSL Expiry
• Email Validator & MX Record Checker – Bulk, Disposable, Role
• Bulk PageSpeed Insights, Lighthouse & Core Web Vitals
• PDF Extractor & Parser – Bulk PDF to Text with Metadata
• Sitemap Extractor & Bulk URL Status Checker
• Steam Reviews Scraper – Game Reviews & App Details
• Website Technology Detector – Tech Stack, CMS & Analytics
• Website Screenshot API – Bulk Full Page PNG, JPEG & PDF

Integrations

Run it from the Apify API, a schedule, or no-code tools: the Apify apps for Zapier, Make and n8n can start any public actor ("Run Actor") and read its dataset. AI agents can call it through the Apify MCP server.

Ready-made examples

One-click tasks you can run or copy and adapt:

• Technical SEO audit of a small website (30 pages)
• SEO checker for a blog: meta tags, H1 and broken links
• Find missing meta descriptions & title tags on a website
• SEO audit of a documentation site: canonicals, noindex, H1
• SEO audit of a SaaS marketing site (titles, meta, headings)
• Find images missing alt text across a website
• Check Open Graph tags and JSON-LD across a whole website