SEO Site Audit & Broken Link Checker: Technical SEO Crawler

Crawl a whole website and get a technical SEO audit of every page in one table: title and meta description (missing, too short, too long, duplicated), H1 headings, canonical tags, noindex, word count, images without alt text, structured data, response time, redirects and broken links. Each website also gets a site summary: pages per issue, every broken link with how many pages carry it, redirect chains, duplicate titles, orphan pages, pages missing from the sitemap, and which AI crawlers the site blocks. Audit 1 website or 100 in one run.

What does SEO Site Audit do?

• Crawls the site like a search engine: starts at the address you give, follows internal links, then reads the XML sitemap (also gzip and sitemap indexes) and audits the pages only the sitemap lists.
• Audits every page for 21 common technical SEO issues (full list below) and gives the raw values too, so you can apply your own rules.
• Checks every link once: internal links, links to files (PDF, images) and links to other websites, and reports each broken one (404, 410, 5xx, dead domains) on the pages that carry it.
• Compares the crawl with the sitemap: orphan pages (in the sitemap, linked from nowhere), sitemap addresses the crawl never reached, and indexable pages missing from the sitemap.
• AI search readiness: lists the well-known AI crawlers the site's robots.txt blocks, and whether it has an llms.txt file.
• Follows redirects step by step and reports redirect chains (two or more hops).
• Fast and light: plain HTTP requests, no browser, no proxy.
• Polite by design: always obeys robots.txt (including Crawl-delay), waits between requests to each website, slows down when a site asks it to, and never logs in.

Who uses it?

• SEO agencies and freelancers: run a first audit for a new client, then schedule it monthly and compare.
• Website owners and marketers: find broken links, missing titles and descriptions before they cost traffic.
• Developers and QA teams: check a site after a migration or a release for 404s, redirect chains and noindex tags left by mistake.
• Content teams: spot thin pages, duplicate titles and pages no other page links to.

How to use it

1. Click Try for free.
2. In Start URLs, add the home page of each website to audit, for example https://www.example.com.
3. Set Maximum pages per site (the default is 100). You pay only for pages audited.
4. Click Start. Download the results as JSON, CSV or Excel, or use the API. The Pages view lists every page with its issues; the Site summaries view gives one row per website.

Options: turn off Check links for a faster run, turn off Check links to other websites to check only your own links, tick Include subdomains to audit blog.example.com together with example.com, and use Exclude URLs containing to skip sections such as /tag/ or ?sort=.

Use with AI agents (MCP)

AI assistants and AI agents can call this Actor through Apify's MCP server at mcp.apify.com or through the Apify API. Questions it answers well:

• "Does example.com have broken links?" Read brokenLinks in the site summary row: each broken link, its status and how many pages carry it.
• "Which pages of example.com have no meta description?" Read the page rows whose issues contain missing-meta-description.
• "Is example.com ready for AI search?" Read blockedForAiBots and hasLlmsTxt in the site summary row.

Only startUrls is required. Keep Maximum pages per site low (for example 20) for quick answers, and read the site summary row first: it sums up the whole audit in one item.

Input example

{
  "startUrls": [{ "url": "https://www.example.com" }],
  "maxPagesPerSite": 500,
  "checkLinks": true,
  "checkExternalLinks": true,
  "includeSubdomains": false,
  "excludeUrlPatterns": ["/tag/", "?sort="],
  "maxConcurrency": 5
}

Output example

Real rows from a test run on 2 November 2026 (lists shortened). One page row:

{
  "recordType": "page",
  "site": "https://crawlee.dev/",
  "url": "https://crawlee.dev/blog",
  "statusCode": 200,
  "depth": 1,
  "redirectChain": [],
  "title": "Crawlee Blog - learn how to build better scrapers | Crawlee for JavaScript · Build reliable crawlers. Fast.",
  "titleLength": 107,
  "metaDescription": "Guides and tutorials on using Crawlee, the most reliable open-source web scraping and browser automation library for JavaScript and Node.js developers.",
  "metaDescriptionLength": 151,
  "h1": null,
  "h1Count": 0,
  "canonical": "https://crawlee.dev/blog",
  "indexable": true,
  "wordCount": 1573,
  "imagesMissingAlt": 0,
  "internalLinks": 40,
  "externalLinks": 29,
  "inboundLinks": 4,
  "brokenLinks": [],
  "schemaTypes": ["Blog"],
  "responseTimeMs": 988,
  "issues": ["title-too-long", "missing-h1"],
  "crawledAt": "2026-11-02T15:35:46.893Z"
}

One site summary row (a 5-page test, so the crawl stopped at the page limit):

{
  "recordType": "site",
  "site": "https://crawlee.dev/",
  "pagesCrawled": 5,
  "pagesAudited": 5,
  "pagesFailed": 0,
  "issueCounts": { "duplicate-meta-description": 3, "canonical-to-other-url": 1, "title-too-long": 2, "missing-h1": 1 },
  "duplicateTitles": [],
  "duplicateDescriptions": [
    {
      "text": "Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.",
      "urls": ["https://crawlee.dev/", "https://crawlee.dev/js"]
    }
  ],
  "redirectChains": [],
  "brokenLinks": [],
  "linksChecked": 121,
  "linksNotChecked": 8,
  "orphanPages": null,
  "sitemapOnly": null,
  "crawlOnly": [],
  "sitemapUrlCount": 4849,
  "blockedForAiBots": [],
  "hasLlmsTxt": true,
  "stoppedBy": "page-limit",
  "message": "Stopped at the maximum pages per site. Raise it to audit more of the site.",
  "crawledAt": "2026-11-02T15:36:07.603Z"
}

orphanPages and sitemapOnly are filled only when the crawl reached and read every page linked from the start page (they stay empty when it stopped at the page limit, as above, or when a linked page did not answer). crawlOnly stays empty when some sitemap files could not be read. The OUTPUT record of each run gives a short summary per website.

Issues checked

Issue	Meaning
missing-title, title-too-short, title-too-long	No title, or shorter than 30 / longer than 60 characters
duplicate-title	Same title as another indexable page
missing-meta-description, meta-description-too-short, meta-description-too-long	No description, or shorter than 70 / longer than 160 characters
duplicate-meta-description	Same description as another indexable page
missing-h1, multiple-h1	No H1 heading, or more than one
missing-canonical, canonical-to-other-url	No canonical tag, or one that points to another address
noindex	Kept out of search results by a robots meta tag or header
low-word-count	Fewer than 200 words of visible text
images-missing-alt	Images without an alt attribute
broken-links	Links to pages or files that answer 4xx or 5xx, or to dead domains
orphan-page	In the sitemap, but no crawled page links to it
broken-page, server-error	The page itself answers 4xx or 5xx
redirect-chain	Reached through two or more redirects
slow-response	Took more than 3 seconds to load

How much does it cost?

You pay only for pages audited. No start fee. Prices per 1,000 pages, by Apify plan:

Event	Free	Bronze	Silver	Gold and up
Page audited	$6.00	$5.00	$4.50	$4.00

• Free: the site summary rows, pages that answer with an error (404, 500…), and every link check.
• Pages over your maximum, pages blocked by robots.txt and excluded pages are not opened and not charged.
• Set a maximum cost per run and the Actor stops cleanly before going over it; the site summary says so.

Example on the Free plan: auditing a 500-page website costs $3.00; a monthly audit of 10 client sites of 200 pages each costs $12.00.

FAQ

Does it run JavaScript? No. It reads the HTML the server sends, as most search engine crawlers do first. On sites that build their pages with JavaScript in the browser, pages may look empty (no title, few words); that is a real SEO risk worth knowing about.

Why was my site not crawled? The site summary row says why in message: robots.txt does not allow crawlers, the site could not be reached, or it belongs to a large platform that does not allow automated access (social networks, search engines, large marketplaces), which this Actor never crawls.

Why do some links count as "not checked"? Links blocked by the target site's robots.txt, links to large platforms, links that answer 401, 403 or 429, links over the link limits (see Limits) and links left when the run ran short of time are not checked. They are counted in linksNotChecked, never reported as broken.

Are 403 answers broken links? No. 401, 403 and 429 usually mean a login wall, bot protection or a rate limit, so such links are counted as not checked, never reported as broken.

Can I audit a staging site? Only if it is on the public internet with a normal domain name. The Actor does not open private network addresses, IP addresses or non-standard ports, and does not log in.

Limits

• Up to 100 websites per run and 50,000 pages per website.
• Up to 10 pages of one website at a time (default 5), with a pause between requests. A robots.txt Crawl-delay is always obeyed in full, so a website that asks for a long delay gets fewer pages audited before the run timeout.
• Link checks: up to 20 links per audited page (at least 500 per site, at most 50,000), and up to 200 links to any one other website.
• Sitemaps: up to 50 sitemap files and 200,000 addresses per website.
• Pages larger than 5 MB are read up to 5 MB, and up to 5,000 links of one page are followed and checked.
• The crawl stops in time to save its results before the run timeout; for large sites give the run more time.

Is it legal?

Audit your own websites, your clients' websites, or websites whose owners allow it. The Actor reads only public pages, always obeys robots.txt, never logs in and never tries to get past blocking. It keeps only the audit fields shown above: no page text is stored, and email addresses and phone numbers found in titles, descriptions and headings are removed. You are responsible for how you use it and for the terms of the websites you audit.

Questions and feedback

Open an issue on the Issues tab.