SEO Audit Crawler: Site Score, Broken Links, Duplicate Titles

Give it a website and it crawls the pages and checks every page for the SEO basics, then gives each page a 0–100 score and the site a free summary of what to fix first.

Checked on every page:

• Title: missing, too long (over 60 characters), too short, duplicated on other pages
• Meta description: missing, too long or short, duplicated
• Headings: missing H1, more than one H1, H2 count
• Indexing: noindex in meta robots or the X-Robots-Tag header, canonical URL pointing elsewhere or missing
• Broken links: links to pages that return 4xx/5xx or don't load (internal by default, external optional)
• Content: word count and thin pages (under 200 words), images without alt text
• Technical: response time, HTML size, redirects, plain HTTP, mobile viewport tag, lang attribute
• Extras: structured data types (JSON-LD), Open Graph tags, hreflang count, internal and external link counts

Site summary (free): average score, issues ranked by how many pages they affect, every broken link with the page it was found on, duplicate titles and descriptions grouped, non-indexable pages, the slowest and lowest-scoring pages, and whether robots.txt and a sitemap exist.

No browser, no login, no API key: it reads pages the way a search engine bot does, gently (a few requests at a time) and respecting robots.txt.

Who it's for

• SEO consultants and agencies: a quick technical audit for a client or a prospect.
• Site owners and marketers: find broken links and missing titles before Google does.
• Developers: check a site after a migration or redesign; schedule it to catch regressions.

Input

Option	Default	What it does
Website	–	Start page(s); links on the same site are followed
Max pages	100	Stop after this many audited pages
Only / skip URLs containing	–	Limit the audit to e.g. /blog/
Check for broken links	on	Also check linked pages that weren't audited (up to 2,000)
Include links to other sites	off	Broken-link check for outgoing links too
Respect robots.txt	on	Skip pages the site asks bots not to visit
Pages at once	4	Keep it low to be gentle

Output

One row per page:

{
  "url": "https://crawlee.dev/js/docs/guides",
  "status": "ok",
  "score": 86,
  "errors": 0,
  "warnings": 2,
  "title": "Guides | Crawlee for JavaScript · Build reliable crawlers. Fast.",
  "titleLength": 64,
  "indexable": true,
  "brokenLinks": [],
  "issues": [
    { "code": "title_too_long", "severity": "warning", "message": "Title is longer than 60 characters and may be cut off in search results.", "detail": 64 },
    { "code": "duplicate_title", "severity": "warning", "message": "Another page has the same title.", "detail": 10 },
    { "code": "duplicate_meta_description", "severity": "notice", "message": "Another page has the same meta description.", "detail": 15 }
  ]
}

Plus one site_summary row. Pages that return an error appear as http_error rows (not charged) and in the broken-link list of the pages that link to them.

Scoring: each page starts at 100. Each error costs 15 points, each warning 6 and each notice 2. It's a simple, transparent checklist score, not a guess at Google's ranking.

Pricing

You pay per audited page. The site summary and failed pages are free. See the Pricing tab.

Limits, honestly

• Pages are read as HTML without running JavaScript. Sites that build everything in the browser (some single-page apps) will show few links and thin content.
• It checks on-page and technical basics. It doesn't measure backlinks, rankings or keyword volumes. For speed and Core Web Vitals, use Lighthouse Audit.
• Sites behind a bot check or a login can't be audited.

More tools from SwiftKit

• Lighthouse Audit: page speed, Core Web Vitals, accessibility and SEO scores in bulk
• Sitemap Extractor: every URL from a site's sitemaps, with broken-link check
• Tech Stack Detector: what a website is built with
• Website to Markdown for AI: clean page content for LLMs and RAG

Questions?

Open an issue on the Issues tab.