# Website SEO Audit for Agencies: Crawler, Reports & Alerts (`hani-tools/seo-audit-for-agencies`) Actor

Crawl client sites and run 77 SEO checks (titles, canonicals, hreflang, noindex, redirects, broken links, sitemaps, structured data). Get a prioritized fix plan, a white-label HTML report and alerts when SEO-critical things change. Respects robots.txt. Not affiliated with Google.

- **URL**: https://apify.com/hani-tools/seo-audit-for-agencies.md
- **Developed by:** [HaniDev](https://apify.com/hani-tools) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $4.00 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Website SEO Audit for Agencies: On-Page Crawler, Client Reports & Change Alerts

Crawl your clients' websites and get an **on-page SEO audit** with **77 checks**, a **prioritized fix plan**, a **white-label HTML client report** and **alerts when something SEO-critical changes** (a page suddenly set to *noindex*, a canonical that moved, robots.txt blocking the site, a sitemap that disappeared, new broken links).

Built for **SEO and web agencies, freelancers and in-house teams** who look after several sites and want a repeatable, scheduled site audit they can send to clients — without per-seat SaaS pricing.

- ✅ Titles, meta descriptions, headings, canonicals, **hreflang return links**, noindex/X-Robots-Tag, redirects and redirect chains, **broken internal links (with anchor text)**, click depth, orphan pages, **XML sitemap consistency**, structured data (JSON-LD) basics, Open Graph, mixed content, thin content and more
- ✅ **Fix plan** ranked by severity × affected pages × page importance, in plain English with "how to fix"
- ✅ **White-label report** per site (your agency name, client name, logo, colour) + portfolio overview + CSV for Excel
- ✅ **Change tracking** between runs: new/fixed issues and critical changes, optional **webhook** and **fail-the-run** for CI
- ✅ **Pay only for pages that were analysed**: error pages, redirects, robots-blocked and unreachable URLs are free
- ✅ Respects **robots.txt**, rate-limited, honest User-Agent, no proxies. HTML only (no browser), so it is fast and cheap

> **Responsible use:** only audit websites that you own or that you are authorised to audit (for example, your clients' sites). You are responsible for how you use this Actor. robots.txt is always respected and cannot be turned off.

### What does it check?

Severity: **C** critical · **H** high · **M** medium · **L** low · *info* (shown but never counted in the score).

| Area | Checks |
|---|---|
| **Titles & descriptions** | `title-missing` (H), `title-empty` (H), `title-multiple` (H), `title-long` (L), `title-short` (L), `title-duplicate` (M), `desc-missing` (M), `desc-multiple` (M), `desc-long` (L), `desc-short` (L), `desc-duplicate` (M) |
| **Headings** | `h1-missing` (M), `h1-multiple` (L), `heading-skip` (L) |
| **Canonicals** | `canonical-missing` (L), `canonical-multiple` (H), `canonical-relative` (L), `canonical-invalid` (H), `canonical-outside-head` (M), `canonical-to-non200` (H), `canonical-to-redirect` (M), `canonical-to-noindex` (H), `canonical-cross-domain` (M), `canonical-noindex-conflict` (M) |
| **hreflang** | `hreflang-invalid-code` (M), `hreflang-no-self` (M), `hreflang-no-return` (M), `hreflang-to-non200` (M), `hreflang-to-noncanonical` (M), `hreflang-missing-xdefault` (info) |
| **Indexability & robots.txt** | `noindex-meta` (info), `noindex-header` (info), `nofollow-page` (info), `noindex-in-sitemap` (H), `robots-blocks-home` (C), `robots-missing` (L), `robots-sitemap-missing` (L), `robots-unreachable` (H) |
| **Status codes & redirects** | `status-4xx` (H), `status-5xx` (H), `redirect-chain` (M), `redirect-loop` (H), `redirect-temporary` (L), `http-not-redirected` (M) |
| **Internal links & structure** | `link-broken-internal` (H), `link-to-redirect` (L), `link-nofollow-internal` (L), `depth-deep` (M), `orphan-in-sitemap` (M) |
| **XML sitemap** | `sitemap-url-non200` (H), `sitemap-url-redirect` (M), `sitemap-url-noncanonical` (M), `sitemap-url-blocked-by-robots` (H), `indexable-not-in-sitemap` (L), `sitemap-too-large` (H), `sitemap-missing` (L), `sitemap-error` (M) |
| **Content** | `thin-content` (M), `js-dependent-suspect` (M) |
| **Images, social & structured data** | `img-alt-missing` (M), `og-missing` (L), `og-image-relative` (L), `twitter-card-missing` (L), `jsonld-parse-error` (H), `jsonld-no-context` (M), `jsonld-no-type` (M), `jsonld-basic-missing` (L), `microdata-present` (info) |
| **Security, weight & basics** | `mixed-active` (H), `mixed-passive` (M), `html-heavy` (L), `html-truncated` (L), `resources-many` (L), `response-slow` (L), `viewport-missing` (M), `lang-missing` (L), `charset-missing` (L) |

Rules marked as **heuristics** in the report (title/description length, thin content, page weight, response time, JavaScript-dependence) are rules of thumb, not search-engine requirements. Google, for example, states there is no fixed limit for title length; titles are shortened to fit the device width.

Site-wide checks include: robots.txt (missing, blocking the home page for search engines, no `Sitemap:` line, unreachable), sitemap (missing, unreadable, over 50,000 URLs / 50 MB, URLs that are broken, redirecting, non-canonical, noindex or blocked by robots.txt, indexable pages missing from it) and whether `http://` redirects to `https://`.

### Output

| Where | What |
|---|---|
| **Dataset** (one row per URL) | status, HTTP code, redirect chain, title/description (+ length, approx. pixel width), H1s, heading outline, canonical, robots directives, indexable, in-sitemap, hreflang, word count, images, Open Graph, JSON-LD types, link counts, broken links with anchor text, inlinks, mixed content, HTML size, response time, issues\[], `charged` flag |
| **Key-value store** `REPORT-<site>.html` | Self-contained white-label client report: health score, critical changes, fix plan, site-wide checks, duplicate titles, all URLs |
| `PORTFOLIO.html` | One-page overview of all sites in the run |
| `ISSUES-<site>.csv` | One row per issue, ready for Excel / Google Sheets |
| `summary.md` | Short Markdown summary for Slack, email or an AI assistant |
| `OUTPUT` | Machine-readable run result for CI and automations (status, critical change count, report URLs) |
| Dataset `seo-audit-site-summaries` | One row per site and run (health score, issue counts, fix plan, sitemap consistency, depth histogram, changes). This named dataset keeps the history across runs. |

**Health score (0–100)** = share of indexable pages that have no critical or high-severity issue. If there is a site-wide critical issue (e.g. robots.txt blocks the home page), the score is capped at 49. "Indexable" = HTML page with HTTP 2xx, not noindex, canonical pointing to itself or absent.

### Change alerts

Turn on **Compare with the previous audit** (default) and schedule the Actor (e.g. weekly per client). Each site keeps a baseline in a named key-value store (`seo-audit-baselines-<baselineName>`). The next run reports:

- **Critical changes**: page was indexable and is now *noindex*; page was 200 and now returns 4xx/5xx; canonical changed (or moved to another domain); robots.txt now blocks pages that were indexable; the XML sitemap is gone; a title was removed; at least *N* new broken internal links.
- New and fixed issues, pages whose title/description/H1/canonical changed, new pages and pages that are gone.

Pages that a run did not reach (page limit, spending limit) are reported as *not re-checked*, never as removed, so a partial run does not cause false alarms. If **Alert webhook URL** is set, a JSON payload is POSTed when there is at least one critical change. **Fail the run on critical changes** makes the run fail, which is handy in deploy pipelines.

### Sample report

[Open the full sample HTML report](https://api.apify.com/v2/key-value-stores/afojEahUKRNXEDyiP/records/seo-audit-sample-report.html) – this is the white-label report the Actor saves for each site.

![Sample client report for choosealicense.com (25 pages)](https://api.apify.com/v2/key-value-stores/afojEahUKRNXEDyiP/records/seo-audit-sample-report.png)

### Pricing (pay per event)

| Event | When it is charged | Price |
|---|---|---|
| `page-audited` | an HTML page returned HTTP 2xx and was analysed | **$0.006** (Free/Bronze), $0.005 (Silver), $0.004 (Gold and above) |
| `site-report` | a site with at least one analysed page got its white-label report, fix plan and change comparison | **$0.05** (Free/Bronze), $0.04 (Silver), $0.03 (Gold and above) |

**Free:** 4xx/5xx pages, redirects, robots.txt-blocked URLs, non-HTML files, blocked/unreachable URLs, extra target checks (HEAD requests), the portfolio, CSV, Markdown summary, OUTPUT and webhooks. Apify also charges its standard small Actor start fee.

Examples: one site with 25 pages ≈ $0.20 · 100 pages ≈ $0.65 · ten client sites × 100 pages per month ≈ $6.50.

**Spending limit:** if you set a maximum cost per run, the Actor keeps enough budget for each site's report, stops requesting new pages when the next one could not be paid for, and still delivers the report for the pages it analysed, clearly marked *partial audit*.

### Input example

```json
{
  "startUrls": ["https://www.client-one.com/", "https://www.client-two.com/"],
  "maxPagesPerSite": 100,
  "excludeUrlPatterns": ["/tag/", "?sort="],
  "compareWithPrevious": true,
  "baselineName": "weekly",
  "webhookUrl": "https://hooks.example.com/seo-alerts",
  "brandName": "Acme Digital",
  "clientName": "Client One",
  "logoUrl": "https://acme.example/logo.png",
  "accentColor": "#2255aa"
}
```

Useful options: **Sample N pages per section** (`samplePerPathPattern`) to cover big sites evenly, **include/exclude patterns**, **max click depth**, **thresholds** (e.g. `{"titleMax": 65, "thinWords": 150}`), **include subdomains**.

### How it crawls (and how it treats your clients' servers)

- Plain HTTP requests (no browser). The User-Agent identifies the Actor: `Mozilla/5.0 (compatible; AgencySEOAudit/1.0; +<Actor page URL>)`. No proxies, no User-Agent rotation, no CAPTCHA or login bypass.
- **robots.txt is always respected** for the `AgencySEOAudit` token (falling back to `*` rules). If robots.txt returns a server error, the site is not crawled. A `Crawl-delay` up to 10 s is honoured.
- One request at a time per site, at least 1 s apart by default (minimum 0.5 s). HTTP 429/503 responses are retried after `Retry-After`. After 5 failures in a row the site is skipped (shown as *blocked/unverified*, not charged).
- Pages are discovered from the start URL and the XML sitemap (breadth-first, so click depth is real). About 20% of the page limit is reserved for sitemap-only URLs so orphan and sitemap checks have data.
- Every redirect hop is checked; private/internal network addresses are refused.
- Page content is analysed in memory. Pages are not stored except for the fields listed in *Output*.

### Limitations

- **JavaScript is not executed.** Content that only appears after JavaScript runs is not seen. Pages that look like a JavaScript app shell are flagged (`js-dependent-suspect`) so you know to check them with a rendering crawler. A JavaScript rendering mode is planned.
- Response time is a single measurement and includes network latency; this is not a page-speed test.
- Structured-data checks are basic and do **not** claim rich-result eligibility.
- Click depth and orphan detection only see the pages that were crawled; with a page limit they are estimates (labelled "possible").
- No keyword rankings, backlinks or traffic data (these need third-party data).

### FAQ

**Can I audit a site I do not manage?** Only if you have permission. The Actor respects robots.txt and rate limits, but you are responsible for having the right to audit the site.

**Why is a page missing from my results?** It may be blocked by robots.txt (listed as *skipped (robots.txt)*), outside the page/depth limit, filtered by your patterns, or not linked from any crawled page and not in the sitemap.

**Is this a Google tool?** No. Not affiliated with or endorsed by Google. It uses no Google APIs. Googlebot rules in robots.txt are evaluated locally to report SEO problems; nothing is requested as or pretending to be Googlebot.

**Can I use it from my own code or an AI agent?** Yes: run it via the Apify API, read `OUTPUT` and `summary.md` from the key-value store.

***

Not affiliated with or endorsed by Google. No Google APIs are used.

# Changelog

This Actor's version history is a separate document: https://apify.com/hani-tools/seo-audit-for-agencies/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

One start URL per site (usually the home page). Several sites can be audited in one run; each gets its own report. Only audit sites you own or are authorised to audit (e.g. client sites). robots.txt is always respected.

## `maxPagesPerSite` (type: `integer`):

Upper limit of HTML pages analysed (and billed) per site. Error pages, redirects and robots-blocked URLs are free and do not count.

## `useSitemap` (type: `boolean`):

Read the sitemap (from robots.txt, or /sitemap.xml) to find pages and to check that it matches the site (broken, redirecting, noindex or orphan URLs).

## `sitemapUrls` (type: `array`):

Use these sitemaps instead of auto-discovery.

## `maxDepth` (type: `integer`):

Do not follow links more than this many clicks from the start URL.

## `samplePerPathPattern` (type: `integer`):

0 = off. For example 5 analyses at most 5 pages per first path segment (/blog/*, /products/*), so big sites are covered evenly within the page limit.

## `includeUrlPatterns` (type: `array`):

Plain text, \* wildcards or /regex/. Start URLs are always included.

## `excludeUrlPatterns` (type: `array`):

Plain text, \* wildcards or /regex/, e.g. /tag/, ?sort=, /page/.

## `includeSubdomains` (type: `boolean`):

Treat blog.example.com etc. as part of example.com.

## `checkCanonicalTargets` (type: `boolean`):

Send a light HEAD request to targets that were not crawled (up to the limit below) to detect broken links and bad canonicals. These requests are free.

## `maxExtraChecks` (type: `integer`):

Upper limit for the free HEAD checks of uncrawled link/canonical/hreflang targets. Default: 2 × max pages per site, at most 200.

## `thresholds` (type: `object`):

Override heuristic limits: titleMax, titlePxMax, titleMin, descMax, descMin, thinWords, maxDepth, htmlMaxKb, maxResources, slowTtfbMs, jsSuspectWords.

## `compareWithPrevious` (type: `boolean`):

Stores a baseline per site and reports new/fixed issues and critical changes (noindex added, page now 4xx/5xx, canonical changed, robots.txt newly blocking, sitemap gone, title removed, new broken links).

## `baselineName` (type: `string`):

Use different names to keep separate histories (e.g. one per client or schedule).

## `webhookUrl` (type: `string`):

POSTs a JSON summary when at least one critical change is found (Slack/Zapier/Make compatible endpoint).

## `failRunOnCritical` (type: `boolean`):

Useful in CI/deploy pipelines. Results are still saved.

## `newBrokenLinksThreshold` (type: `integer`):

Alert when at least N new broken internal links.

## `reportEnabled` (type: `boolean`):

One self-contained HTML report per site (billed as one site-report event) plus a free portfolio overview.

## `brandName` (type: `string`):

Shown in the report header and footer.

## `clientName` (type: `string`):

Shown in the report header.

## `logoUrl` (type: `string`):

https:// URL of your logo (PNG/SVG).

## `accentColor` (type: `string`):

Hex colour such as #2255aa.

## `perHostDelaySecs` (type: `number`):

Politeness delay; minimum 0.5 s. A longer robots.txt Crawl-delay (up to 10 s) is honoured automatically.

## `requestTimeoutSecs` (type: `integer`):

Give up on a request after this many seconds (not charged).

## `maxHtmlMb` (type: `integer`):

Pages larger than this are cut at this size and flagged.

## `maxConcurrentSites` (type: `integer`):

Each site is still crawled one request at a time.

## Actor input object example

```json
{
  "startUrls": [
    "https://crawlee.dev/"
  ],
  "maxPagesPerSite": 10,
  "useSitemap": true,
  "maxDepth": 10,
  "samplePerPathPattern": 0,
  "includeSubdomains": false,
  "checkCanonicalTargets": true,
  "thresholds": {
    "titleMax": 60,
    "descMax": 160,
    "thinWords": 200,
    "maxDepth": 4
  },
  "compareWithPrevious": true,
  "baselineName": "default",
  "failRunOnCritical": false,
  "newBrokenLinksThreshold": 3,
  "reportEnabled": true,
  "accentColor": "#2255aa",
  "perHostDelaySecs": 1,
  "requestTimeoutSecs": 20,
  "maxHtmlMb": 5,
  "maxConcurrentSites": 3
}
```

# Actor output Schema

## `results` (type: `string`):

One row per URL: status, indexability and the SEO issues found on the page.

## `portfolio` (type: `string`):

Overview of all sites with links to each site's client report.

## `summary` (type: `string`):

Short Markdown summary of the run.

## `siteSummaries` (type: `string`):

One record per site: scores, issues, changes and report URL.

## `files` (type: `string`):

All key-value store records, including the per-site HTML reports and the issue CSV files.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawlee.dev/"
    ],
    "maxPagesPerSite": 10,
    "thresholds": {
        "titleMax": 60,
        "descMax": 160,
        "thinWords": 200,
        "maxDepth": 4
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("hani-tools/seo-audit-for-agencies").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://crawlee.dev/"],
    "maxPagesPerSite": 10,
    "thresholds": {
        "titleMax": 60,
        "descMax": 160,
        "thinWords": 200,
        "maxDepth": 4,
    },
}

# Run the Actor and wait for it to finish
run = client.actor("hani-tools/seo-audit-for-agencies").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawlee.dev/"
  ],
  "maxPagesPerSite": 10,
  "thresholds": {
    "titleMax": 60,
    "descMax": 160,
    "thinWords": 200,
    "maxDepth": 4
  }
}' |
apify call hani-tools/seo-audit-for-agencies --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,hani-tools/seo-audit-for-agencies"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/VhM2D7SNPuZ11iQ48/builds/fy4rkJTubHKTmdZbh/openapi.json