SEO Sitemap & Indexability Auditor

Comprehensive SEO audit tool that crawls websites to detect broken links, indexability issues, meta tag problems, and structured data gaps. Built for SEO agencies, website owners, and SaaS founders.

What It Extracts

• Broken links: Internal and external broken links with status codes
• Meta audit: Title tags, meta descriptions, heading structure (H1/H2)
• Indexability: Canonical tags, robots directives, noindex/nofollow flags
• Schema markup: JSON-LD presence, schema types detected
• Social previews: OpenGraph tags, Twitter Card tags
• Sitemap analysis: Automatic sitemap discovery and URL extraction

Why It's Better

Combines sitemap discovery, broken link detection, and full meta/schema audit in one run. Automatically discovers sitemaps from robots.txt and crawls up to 500 pages. Generates a priority fixes report.

Input

Field	Type	Default	Description
startUrls	string[]	required	URLs or sitemap URLs to audit
maxPages	integer	50	Maximum pages to crawl (1-500)
checkExternalLinks	boolean	false	Check external links for broken status
includeMetaAudit	boolean	true	Audit titles, descriptions, headings
includeSchemaAudit	boolean	true	Check JSON-LD presence
includeSocialPreviewAudit	boolean	true	Check OG/Twitter tags

Output Example

{
  "url": "https://example.com/pricing",
  "statusCode": 200,
  "title": "Pricing Plans",
  "description": "Choose the right plan for your team",
  "canonical": "https://example.com/pricing",
  "robots": null,
  "hasNoindex": false,
  "h1Count": 1,
  "h2Count": 4,
  "internalLinks": 15,
  "externalLinks": 3,
  "hasSchema": true,
  "schemaTypes": ["Product", "Offer"],
  "hasOgTags": true,
  "hasTwitterCards": true,
  "issues": [],
  "brokenLinks": [],
  "crawledAt": "2025-01-15T10:30:00Z"
}

Use Cases

• Client SEO audits: Generate comprehensive audit reports for clients
• Pre-launch checks: Verify SEO setup before site launch
• Broken link monitoring: Regular checks for broken links
• Schema validation: Ensure structured data is present on key pages
• Migration audits: Verify SEO elements after site migration

PPE Pricing

Event	Description	Suggested Price
url-checked	One page crawled and analyzed	$0.002
site-audited	Complete site audit report generated	$0.01

Limitations

• Only crawls pages reachable from provided URLs
• External link checking is optional (adds time)
• JavaScript-rendered content may not be fully captured
• Rate limited to respect target sites

Legal/Ethical Use

This actor crawls publicly accessible pages. It respects robots.txt directives. Users are responsible for compliance with applicable laws and website terms of service.

Local Run

cd actors/seo-sitemap-indexability-auditor
apify run --input-file .actor/sample_input.json

Deploy

cd actors/seo-sitemap-indexability-auditor
apify push

FAQ

Q: How does sitemap discovery work? A: It checks robots.txt for Sitemap directives, then tries common sitemap locations (/sitemap.xml).

Q: Can it crawl JavaScript-rendered pages? A: It uses CheerioCrawler which handles most server-rendered pages. For heavy JS sites, consider using a browser-based crawler.

Q: What's the maximum number of pages? A: Up to 500 pages per run. Set maxPages to control crawl depth and cost.

Tags

SEO audit, broken link checker, sitemap analyzer, indexability audit, meta tag checker, schema validator, OpenGraph audit, technical SEO, website health check