Website SEO Audit & Broken Links — Agency Reports

Turn an authorized client website audit into a branded report and a practical fix queue. Find technical SEO issues and broken links, inspect priority landing pages, and compare repeat audits across the same URLs and settings.

Built for SEO agencies, consultants, and website teams serving US businesses. Reports include self-contained HTML, optional US Letter PDF, page-level JSON, and spreadsheet-ready CSV exports. No proxy or AI subscription is required for the standard audit.

What your agency gets

• Branded client delivery: agency name, client or campaign title, and an optional HTML accent color.
• Grouped fix queue: fixes ranked by severity, business-page exposure, and affected-page count, with a suggested owner role and all affected URLs.
• Priority landing-page checks: queue pricing, service, product, and contact pages first. Use URL list mode to monitor selected pages after a release.
• Sitemap discovery: optionally audit sitemap pages that navigation may not link to, within the same page budget.
• Reliable change reports: compare the exact audited URL set and audit settings. Detect new, resolved, unchanged, and materially changed findings, including evidence or severity changes.
• Explicit coverage: see skipped/failed pages, selected URLs actually audited, unchecked links, and whether the budget limited the audit.

Business priority uses your selected URLs and transparent URL-path hints. It does not estimate traffic, conversions, or lost revenue.

Checks included

HTTP status, redirects, broken internal links, optional external-link checks, titles, meta descriptions, headings, canonicals, Googlebot and general indexability directives, duplicate metadata on indexable pages, sitemap membership, missing image alt attributes, mixed content, Open Graph metadata, JSON-LD syntax, real TLS certificate expiry, and selected security headers.

Optional Google PageSpeed Insights provides lab and available field metrics. The crawler reads server HTML using Crawlee BasicCrawler and Cheerio; it does not render JavaScript. Title and description length checks are editorial hints, not fixed Google ranking requirements.

Missing or failed PageSpeed measurements are explicitly skipped; unavailable scores and timings stay unknown. TLS and PageSpeed checks run after pages are collected, so an optional inspection failure preserves page results. Network deadlines cover DNS, redirects, and streamed responses, including servers that keep sending small chunks.

Start with a safe one-page demo

Run the public example, or use this input:

{
  "startUrl": "https://example.com",
  "confirmAuthorizedUse": false,
  "maxPages": 1,
  "maxCrawlDepth": 0,
  "checkExternalLinks": false,
  "maxInternalLinks": 25,
  "compareWithPreviousRun": false,
  "agencyName": "Your agency",
  "clientName": "One-page demo",
  "brandColor": "#2563eb",
  "generateHtmlReport": true,
  "generatePdfReport": true,
  "generateCsvReport": true
}

For a real client website, replace the URL and confirm authorization you already have.

Agency workflows

Weekly client retainers. Save an Apify task for each authorized client. Keep crawl settings stable, enable baseline comparison, and schedule recurring runs. Use the report and grouped CSV queue to plan client work. Each client has a separate baseline namespace, even if multiple clients share a baseline label.

Landing-page monitoring. Set crawlMode to url_list, provide priorityUrls, and keep maxPages large enough to cover the starting page plus all selected pages. Only those pages are audited; optional bounded status checks may still inspect their link targets.

Migration and release QA. Audit priority pages before and after a deployment. Use failOnSeverity to fail on errors or critical findings after reports are saved. Changing the audited URLs or audit settings establishes a fresh baseline instead of claiming omitted findings were resolved.

Sitemap review. Enable crawlSitemap in website mode to queue sitemap URLs after the starting and selected priority pages. A finite crawl cannot prove a page is orphaned; this option helps inspect URLs absent from the sampled navigation.

Price and examples

The current standard event price is $0.005 per audited, non-blocked page ($5 per 1,000 pages), plus $0.00005 per Actor-start event. The start event has a minimum of one unit and scales with Actor memory. Check Apify Console for the authoritative price and applicable platform or plan charges.

Workflow	Page events	Standard page-event subtotal
One 25-page client audit	25	$0.125
One 100-page client audit	100	$0.50
Ten 100-page clients, four runs each	4,000	$20.00
Fifty selected pages checked daily for 30 days	1,500	$7.50

Branded reports and fix queues have no additional Actor event fee. Blocked HTTP 401/403/429/451 responses, robots-denied URLs, failed requests without a useful response, and duplicate final pages are not paid page records. A useful HTTP 404/5xx audit can be billed because the status and findings are the result.

Set maximum cost in Apify run options. Page requests are limited to the page events the budget can cover. Reports describe stored results. Additional link checks have a start-time budget of 10 seconds per collected page, with a 10-second minimum and 180-second maximum; in-flight bounded requests can finish afterwards. Unchecked targets are disclosed. Slow links and PageSpeed attempts can increase resource use.

Input reference

Field	Default / limit	Purpose
startUrl	Required HTTP/HTTPS URL	Authorized starting page
confirmAuthorizedUse	false	Required for real targets; example.com demo is exempt
maxPages	25 / max 250	Maximum page requests; skips or duplicate redirects can yield fewer records
maxCrawlDepth	3 / max 10	Link depth in website mode
maxLinksPerPage	100 / max 500	Bounded link extraction
crawlMode	website	Website crawl or url_list for selected pages
priorityUrls	Empty / max 50	Absolute URLs inside the authorized starting domain
crawlSitemap	false	Seed sitemap pages in website mode
maxInternalLinks	200 / max 2,000	Additional internal status checks beyond crawled pages
checkExternalLinks	false	Enable bounded external status checks
maxExternalLinks	250 / max 2,000	Unique external-check cap
agencyName, clientName	Empty / max 100 characters	Plain-text client report branding
brandColor	Blue fallback	HTML accent in #RRGGBB format
includePerformance	false	Enable Google PageSpeed Insights
pageSpeedApiKey	Optional secret	User-provided Google API key; excluded from outputs
maxPerformancePages	1 / max 5	Maximum PageSpeed attempts, including failed attempts
compareWithPreviousRun	true	Enable private baseline comparison
baselineKey	Empty / max 48 characters	Separate audit profiles within each client domain
generateHtmlReport	true	Save branded HTML report
generatePdfReport	false	Save US Letter PDF report
generateCsvReport	true	Save detailed issues and grouped fix queue CSVs
failOnSeverity	none	Optional error or critical threshold

Output and integrations

The dataset contains one page-audit record per stored useful response, including final URL, status, metadata, indexability, link findings with anchor text when available, score, business context, and evidence-backed issues.

The run Key-Value Store contains:

• OUTPUT: totals, change counts, coverage, branding, fix queue, CI status, and artifact URLs.
• FIX_QUEUE: grouped JSON fixes with rank, priority, recommended owner, affected URLs, and rationale.
• report.html: branded client report when HTML or PDF output is enabled.
• report.pdf: optional PDF report.
• issues.csv: formula-safe detailed findings when CSV output is enabled.
• fix-queue.csv: formula-safe grouped work queue when CSV output is enabled.

Use the Apify API, MCP server, or Apify integrations in n8n, Make, Zapier, spreadsheets, and your reporting system. Read OUTPUT for the run decision and FIX_QUEUE to create internal work items. The Actor does not automatically send reports, create tickets, or contact clients.

How comparisons stay reliable

Baselines store minimized issue metadata, fingerprints, evidence hashes, and a coverage signature; they never store raw HTML. A comparable run must have the same audited URLs, checked link targets, and relevant audit settings. Branding changes do not invalidate the comparison.

The previous baseline is preserved when requests fail, pages are skipped or blocked, the link-check time limit is reached, the page allowance is reduced, or output is trimmed by the budget. Legacy baselines establish a fresh baseline once under the new coverage rules. A clean bounded report describes only its audited scope.

OUTPUT.baselineSaved confirms whether the current results were persisted as a baseline. A baseline write failure leaves this flag false and includes an explicit reason in the JSON summary.

Unresolved HTTP 3xx page responses receive informational PAGE_REDIRECT_UNRESOLVED findings and preserve the prior baseline. coverage.unresolvedRedirectPages counts these responses. Unresolved redirects while fetching robots.txt do not establish crawl permission; successfully followed redirects and absent robots files retain their normal behavior.

Sitemap retrieval failures, invalid XML, excluded declared sitemaps, and discovery limits preserve the prior baseline and suppress missing-from-sitemap findings. Already discovered members remain available for audit. coverage.sitemapDiscoveryComplete, coverage.failedSitemaps, and coverage.sitemapLimitReached disclose discovery coverage; page records include sitemapContext.discoveryComplete. A false inSitemap value is inconclusive when discovery is incomplete. A speculative /sitemap.xml returning 404 or 410 is allowed to be absent; a missing declared sitemap is incomplete discovery.

Storage API rate limits are separate from the run spending cap. coverage.failedPageWrites counts page write/charge operations that did not return a successful SDK confirmation. Reports retain other confirmed rows, disclose the incomplete operation, and preserve the prior baseline. A failed SDK operation can have an uncertain storage or charging outcome; inspect the dataset and run log. The Actor does not retry those page operations itself.

Timeouts, DNS/transport failures, and unresolved redirects are unverified link observations, not proof of a broken URL. They do not create broken-link errors or inflate broken-link counts. Restricted and unverified link checks preserve the prior baseline. Malformed hrefs remain errors, and observed non-restricted HTTP 4xx/5xx responses remain broken-link findings. coverage.unverifiedLinks and linkSummary.unverifiedLinksCount disclose failed verification; the compatibility brokenLinks array includes these observations with a findingType field.

Coverage signatures include whether each page received HTML content checks. A switch between successful HTML and status-only responses establishes a fresh comparison scope instead of claiming unavailable metadata findings were resolved. auditScope and coverage.htmlPagesAudited identify the checks applied.

Exactly spending the allowance on the final requested page does not make an otherwise complete audit partial. Baselines are preserved when the allowance actually reduces coverage or prevents results from being saved.

Authorization, controls, and limitations

Audit websites you own or have explicit prior authorization to inspect. confirmAuthorizedUse confirms existing permission. Recursive crawling stays in the starting registrable domain; external targets receive bounded status checks only when enabled.

Robots.txt applies before page requests, redirect hops, sitemap requests, and link checks. A missing robots file permits crawling; failed or restricted robots retrieval conservatively denies requests. Private infrastructure, URL credentials, unsafe ports, unsafe DNS results, and unsafe redirects are blocked. HTTP bodies, sitemap traversal, depth, and link counts are bounded.

Sitemap locations decode escaped XML characters and numeric references, while custom entity declarations are rejected. Duplicate sitemap-index entries use one traversal slot. URL normalization preserves the order of repeated functional query values; reversing them can identify a different page. TLS inspection uses the same permitted port as the HTTPS URL, and initial DNS validation has a 15-second deadline.

The crawl queue uses the Actor's normalized URL as its explicit request key. Distinct paths such as /catalog and /catalog/ remain separate for both supplied URLs and discovered internal links.

Hosted clients use private public-suffix boundaries, so separate GitHub Pages, Blogspot, and S3 tenants have separate crawl scopes and baseline namespaces. The confirmation checkbox for one client does not authorize crawling another tenant on the same hosting service.

HTML parsing honors the first document base for relative links, canonical URLs, and resource URLs. Standard metadata names and rel tokens ignore ASCII case. A malformed canonical is reported as a parsing error; an external-domain canonical is an informational observation whose intent and content relationship need review. Non-HTML responses receive status and TLS observations rather than HTML SEO checks.

HTML classification matches the exact Content-Type MIME essence; parameters and similarly named subtypes cannot trigger HTML rules or PageSpeed calls. Page decoding retains decompressed bytes so byte order marks, HTTP charset declarations, and HTML encoding declarations can preserve metadata correctly. Documents without an encoding declaration retain the UTF-8 fallback. Raw response bytes are not stored in page audit rows or baselines.

Supported Content-Encoding stacks are decoded in reverse application order, with a maximum of three layers. The body size cap applies to the received body and every intermediate/final decompressed output. Unsupported or excessive stacks fail the bounded fetch rather than returning partially decoded content.

Robots and sitemap responses use the same bounded HTTP decompression. Sitemap files may also contain one gzip file layer after transport decoding, including .xml.gz files and gzip files served without that suffix. Decoded sitemap sizes count toward the graph byte budget. Unreadable or oversized robots responses conservatively deny crawling; unreadable sitemaps remain explicitly incomplete.

This HTTP-only tool cannot evaluate content added only by JavaScript. Unchecked links are not verified healthy. Security-header, image-alt, and TLS observations are not compliance, accessibility, or penetration-test certifications. Rankings and business outcomes are not guaranteed. See SECURITY.md and DOMAIN-REMOVAL.md for controls and owner opt-out requests.

TLS timeouts and connection failures are reported as informational TLS_UNVERIFIED findings, with validity left unknown. Runs containing unverified HTTPS certificate inspections preserve the prior baseline so existing certificate findings cannot be reported as resolved without evidence. Certificate-validation errors remain critical. Preview restrictions such as max-image-preview: none are separate from noindex. Additional internal-link budgets count only targets not already observed during the crawl.