# Website SEO Audit Crawler (`teddythebuilder/website-seo-audit`) Actor

Crawl any website and audit every page for technical SEO: status codes, redirects, broken links, titles, meta descriptions, H1s, canonicals, noindex, hreflang, JSON-LD schema, sitemap coverage and duplicates. No browser, no login. Export JSON/CSV/Excel or call via API.

- **URL**: https://apify.com/teddythebuilder/website-seo-audit.md
- **Developed by:** [Teddy Wasserman](https://apify.com/teddythebuilder) (community)
- **Categories:** SEO tools, Developer tools, Lead generation
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.10 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Website SEO Audit Crawler do?

**Website SEO Audit Crawler** is a fast, low-cost **technical SEO audit tool and site crawler** that runs in the cloud. Give it a website and it crawls the pages the way a search engine bot does, then returns one clean row per page with everything an SEO audit needs: **HTTP status codes, redirect chains, broken links (internal and external), title and meta description length, H1 headings, canonical tags, noindex and X-Robots-Tag directives, hreflang, Open Graph tags, JSON-LD structured data types, word count, images missing alt text, mixed content, sitemap coverage and duplicate titles and descriptions**. Each page also gets a list of machine-readable issue codes and a 0–100 SEO score.

On top of the per-page rows, every run writes a **site summary** (the `SUMMARY` record) with issue counts by severity, groups of duplicate titles and meta descriptions, all broken link targets, orphan sitemap URLs and pages blocked by robots.txt. That is your audit report in one JSON file.

It needs no browser, no login and no API key. It reads `robots.txt` and the XML sitemaps itself, so a 500-page audit usually finishes in a minute or two and costs about $4.

Typical uses:

- **SEO agencies:** run a technical audit for a new client in minutes, or schedule a weekly crawl of every client site and catch new 404s, noindex mistakes and missing titles before rankings drop.
- **In-house SEO and content teams:** find thin pages, duplicate titles, pages missing from the sitemap and images without alt text.
- **Developers and QA:** check a site after a migration or release for broken internal links, redirect chains and pages that suddenly return 5xx.
- **Lead generation:** audit prospects' websites and open the sales conversation with concrete issues ("your sitemap lists 40 URLs that return 404").

### Why use Website SEO Audit Crawler?

- **One row per page, ready for a spreadsheet.** Flat columns such as `statusCode`, `titleLength`, `metaDescriptionLength`, `h1Count`, `canonicalUrl`, `indexable`, `inSitemap`, `brokenLinksCount` and `seoScore` work directly in Google Sheets, Excel or Looker Studio.
- **Issues are already diagnosed.** Instead of raw HTML you get issue codes such as `missing_title`, `duplicate_meta_description`, `broken_internal_links`, `redirect_chain`, `canonical_points_elsewhere`, `noindex_in_sitemap` and `not_in_sitemap`, each with a severity (error, warning, notice).
- **Broken link checker included.** Every internal link target is verified. External links are checked too (HEAD request with a GET fallback), and link checks are **free**: you only pay per audited page.
- **Sitemap coverage and orphan pages.** The crawler reads the sitemaps listed in `robots.txt` (including sitemap indexes and gzipped sitemaps), audits orphan URLs that are in the sitemap but not linked from the site, and flags indexable pages that are missing from the sitemap.
- **Three modes.** Crawl a whole site, audit only the sitemap URLs, or audit an exact list of URLs (a bulk URL status and metadata checker).
- **Polite and safe.** `robots.txt` rules and crawl-delay are always respected, concurrency is limited, and the crawl stays on the site you give it.
- **Schedulable and API-ready.** Run it daily or weekly with Apify schedules, connect it to Make, Zapier or n8n, or let an AI agent call it through the Apify API or MCP server.
- **Never loses results.** The crawler stops itself before the run timeout and always saves the pages it has audited plus the summary.

### What data can you extract?

| Field | Example | Description |
|---|---|---|
| `url` | `https://www.example.com/blog/` | The audited URL |
| `statusCode` | `200` | First HTTP status (e.g. `301` for a redirect) |
| `finalUrl` / `finalStatusCode` | `https://www.example.com/blog/`, `200` | Where redirects end and the status there |
| `redirectChain` | `[{"url": "http://example.com/", "statusCode": 301}]` | Each hop of a redirect |
| `responseTimeMs` / `pageSizeKb` | `412`, `88.4` | Server response time and HTML size |
| `depth` / `foundOn` | `2`, `https://www.example.com/` | Clicks from the start URL and the page that linked to it (or `sitemap`) |
| `title` / `titleLength` | `Blog - Example`, `14` | `<title>` and its length |
| `metaDescription` / `metaDescriptionLength` | `News and guides...`, `132` | Meta description and its length |
| `h1` / `h1Count` / `h2Count` | `["Our blog"]`, `1`, `6` | Headings |
| `canonicalUrl` | `https://www.example.com/blog/` | `<link rel="canonical">` |
| `metaRobots` / `xRobotsTag` | `noindex, follow` | Robots directives from HTML and HTTP headers |
| `indexable` | `true` | 2xx, not redirected, no noindex, canonical is self or missing |
| `inSitemap` | `true` | Whether the URL is listed in the site's sitemap |
| `lang` / `viewport` | `en`, `true` | `<html lang>` and mobile viewport tag |
| `hreflang` | `[{"lang": "sv", "url": "https://www.example.com/sv/"}]` | Alternate language versions |
| `openGraph` / `twitterCard` | `{"title": "...", "image": "..."}` | Social sharing tags |
| `jsonLdTypes` / `microdataTypes` | `["Organization", "BreadcrumbList"]` | Structured data (schema.org) types found |
| `jsonLdInvalid` | `0` | JSON-LD blocks that are not valid JSON |
| `wordCount` | `812` | Visible words (scripts and styles removed) |
| `imagesCount` / `imagesMissingAlt` / `imagesMissingAltSample` | `14`, `3`, `[...]` | Image alt text check |
| `mixedContent` | `["http://cdn.example.com/a.js"]` | Insecure resources on HTTPS pages |
| `internalLinksCount` / `externalLinksCount` | `48`, `7` | Outgoing links |
| `brokenLinks` / `brokenLinksCount` | `[{"url": ".../old", "status": 404, "anchorText": "Pricing", "internal": true}]` | Broken link targets on this page |
| `issues` | `["title_too_long", "missing_h1"]` | Issue codes (see the FAQ for the full list) |
| `errorsCount` / `warningsCount` | `0`, `2` | Issue counts by severity |
| `seoScore` | `84` | 0–100 page score: −25 per error, −8 per warning, −2 per notice |
| `error` | `null` | Fetch error (timeouts, DNS, redirect loops) |
| `checkedAt` | `2026-10-06T12:00:00+00:00` | Timestamp |

The **SUMMARY** record (key-value store) contains: `pagesAudited`, `averageSeoScore`, `statusCodes`, `indexablePages`, `issues` (each issue with severity and number of pages), `duplicateTitles`, `duplicateMetaDescriptions`, `brokenLinks`, `sitemap` (URLs in sitemaps, sitemap URLs not audited), `blockedByRobotsTxt` and `linkTargetsChecked`.

### How to use Website SEO Audit Crawler (step by step)

1. Click **Try for free** (you need a free Apify account; new accounts get monthly free credits).
2. Enter the homepage of the site in **Website or page URLs**, e.g. `https://www.example.com/`.
3. Set **Max pages**. Start with 50–200 to get a feel for the site, then raise it for a full audit.
4. Pick a **Crawl mode**: crawl the site (default), sitemap URLs only, or only the URLs you entered.
5. Click **Start**. Watch the log: it reports sitemap discovery, crawl progress and link checks.
6. Open the **SEO audit** view in the Output tab, sort by `seoScore` or filter by issue, and download as **JSON, CSV, Excel or HTML**. Open the **Site summary** output for the report.

### Input example

```json
{
  "startUrls": ["https://www.example.com/"],
  "maxPages": 500,
  "crawlMode": "crawl",
  "checkExternalLinks": true,
  "excludeUrlPatterns": ["/tag/", "\\?replytocom="],
  "onlyPagesWithIssues": false
}
```

Audit an exact list of landing pages instead:

```json
{
  "startUrls": ["https://www.example.com/pricing", "https://www.example.com/features", "http://example.com/old-page"],
  "crawlMode": "list"
}
```

### Output example

```json
[
  {
    "url": "https://www.python.org/about/",
    "statusCode": 200,
    "finalUrl": "https://www.python.org/about/",
    "redirectChain": [],
    "contentType": "text/html",
    "responseTimeMs": 21,
    "depth": 0,
    "foundOn": null,
    "inSitemap": null,
    "title": "About Python™ | Python.org",
    "titleLength": 26,
    "metaDescription": "The official home of the Python Programming Language",
    "metaDescriptionLength": 52,
    "h1": ["Python is powerful... and fast; plays well with others; runs everywhere; is friendly & easy to learn; is Open."],
    "h1Count": 1,
    "canonicalUrl": null,
    "indexable": true,
    "lang": "en",
    "jsonLdTypes": ["WebSite"],
    "wordCount": 698,
    "imagesMissingAlt": 0,
    "brokenLinksCount": 0,
    "issues": ["title_too_short", "meta_description_too_short", "duplicate_meta_description", "missing_canonical"],
    "errorsCount": 0,
    "warningsCount": 1,
    "seoScore": 86
  },
  {
    "url": "https://www.python.org/psf/",
    "statusCode": 301,
    "finalUrl": "https://www.python.org/psf-landing/",
    "finalStatusCode": 200,
    "redirectChain": [{"url": "https://www.python.org/psf/", "statusCode": 301}],
    "issues": ["redirect"],
    "seoScore": 98
  }
]
```

Fields are shortened here. Every run also saves the site report as the `SUMMARY` record. Results can be exported to CSV, Excel, JSON, XML or HTML, or read via the API.

### How much does it cost?

The actor uses **pay-per-event** pricing, so you only pay for pages that were actually audited.

| Event | Price |
|---|---|
| Actor start | $0.0005 per run |
| Page audited | $0.008 per page ($8 per 1,000 pages) |
| Link checks, sitemap parsing, SUMMARY report | free |
| Pages that could not be reached at all (DNS errors, timeouts) | free |

Higher Apify plans get automatic volume discounts on the per-page price (Bronze −15%, Silver −30%, Gold and above −45%).

**Price change:** the per-page price is $0.008 from **26 October 2026** (it was $0.002 until then). The start fee and the free items are unchanged. Full-site audit tools in the Store charge up to $40 per 1,000 pages.

Worked examples:

- **A 300-page company website:** 300 × $0.008 ≈ **$2.40** per full audit.
- **Weekly audits of 10 client sites with 500 pages each:** 10 × 500 × 4 ≈ 20,000 pages ≈ **$160 per month** (about $88 on Gold-tier plans).
- **Daily check of 50 key landing pages in list mode:** 50 × 30 = 1,500 pages ≈ **$12 per month**.

Platform compute is small because there is no browser: a 500-page crawl typically uses a few cents of compute or less on top. Use **Only output pages with issues** to skip (and not pay for) clean pages, and **Max pages** as a hard cost cap.

### Use cases

- **SEO agency monthly reporting.** Schedule a crawl per client, push the SUMMARY into Google Sheets or Looker Studio, and track `averageSeoScore` and error counts over time.
- **Site migrations.** Before launch, audit the old URLs in list mode and check every one redirects in one hop to a 200 page. After launch, crawl the new site for broken internal links and redirect chains.
- **Content cleanup.** Filter on `thin_content`, `duplicate_title` and `duplicate_meta_description` to find pages to merge, rewrite or noindex.
- **Technical health monitoring.** A daily run with `onlyPagesWithIssues` and a webhook alerts you when pages start returning 4xx/5xx or a release accidentally adds `noindex`.
- **Structured data inventory.** Use `jsonLdTypes` to see which templates carry Product, Article, FAQPage or BreadcrumbList markup and which are missing it.
- **Sales prospecting for web agencies.** Audit a list of prospect homepages in list mode and mention concrete issues in outreach.
- **AI agents.** An agent can call the actor with a URL and reason over the issue codes to suggest fixes.

### Use with AI agents, MCP and the API

Run it from the command line (replace `<YOUR_TOKEN>`):

```bash
curl -X POST "https://api.apify.com/v2/acts/teddythebuilder~website-seo-audit/run-sync-get-dataset-items?token=<YOUR_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://www.example.com/"], "maxPages": 100, "onlyPagesWithIssues": true}'
```

Or from Python with `apify-client`:

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_TOKEN>")
run = client.actor("teddythebuilder/website-seo-audit").call(run_input={
    "startUrls": ["https://www.example.com/"],
    "maxPages": 200,
})
for page in client.dataset(run["defaultDatasetId"]).iterate_items():
    if page["errorsCount"]:
        print(page["url"], page["issues"])
summary = client.key_value_store(run["defaultKeyValueStoreId"]).get_record("SUMMARY")["value"]
print(summary["averageSeoScore"], summary["issues"][:5])
```

AI agents (Claude, ChatGPT, Cursor and others) can discover and run this actor through the **Apify MCP server** (`https://mcp.apify.com`). Ask your agent to "audit example.com for technical SEO issues with the Website SEO Audit Crawler".

### Integrations

Connect the actor to **Make, Zapier, n8n, Google Sheets, Slack, Airbyte** or your own backend with **webhooks**. A common setup: schedule a weekly run, send the SUMMARY to Slack, and append the pages with errors to a Google Sheet for the content team.

### Data source and legality

The actor fetches public web pages exactly like a search engine crawler, identifies itself with a clear user agent (`ApifyWebsiteSeoAudit`), **always respects robots.txt and crawl-delay**, and keeps concurrency low. It collects page metadata and technical SEO signals only. It does not log in, does not bypass paywalls or bot protection, and does not extract personal data. Audit sites you own or are allowed to audit, and keep the crawl size reasonable for small sites.

### Limitations

- **No JavaScript rendering.** The audit sees the HTML the server returns, as most crawlers do on the first pass. Sites that build all content with client-side JavaScript will show low word counts and missing tags. This is also a useful signal, because search engines index the raw HTML first.
- **Bot protection.** Pages behind Cloudflare challenges or similar return 403/429. These statuses are reported but not counted as broken links.
- **Sitemaps** are read up to 30 files and 100,000 URLs per run, with a 15-second discovery budget.
- **Very large sites:** crawl in sections with `includeUrlPatterns`, or raise the run timeout and memory.

### FAQ

**Which issues are detected?**
Errors: `http_4xx`, `http_5xx`, `fetch_failed`, `redirect_loop`, `missing_title`, `broken_internal_links`. Warnings: `title_too_long` (>60 characters), `missing_meta_description`, `meta_description_too_long` (>160), `missing_h1`, `broken_external_links`, `images_missing_alt`, `slow_response`, `mixed_content`, `duplicate_title`, `duplicate_meta_description`, `invalid_json_ld`, `noindex_in_sitemap`, `canonical_to_non_200`, `redirect_chain`. Notices: `title_too_short` (<30), `meta_description_too_short` (<70), `multiple_h1`, `missing_canonical`, `canonical_points_elsewhere`, `noindex`, `thin_content` (<200 words), `missing_lang`, `missing_viewport`, `missing_open_graph`, `not_in_sitemap`, `redirect`, `url_too_long`, `non_html`.

**Does it work on the free Apify plan?**
Yes. A 20-page test run costs about $0.16, well inside the monthly free credits.

**How do I stop it from crawling filters, tags or search pages?**
Add regular expressions to **Exclude URLs matching**, e.g. `/tag/`, `sort=` or `\?page=`.

**Why is a 403 link not reported as broken?**
401, 403, 429 and 999 usually mean the target blocks bots (LinkedIn, some CDNs), not that the page is gone, so they are not counted as broken links.

**Can I audit several sites in one run?**
Yes. Add several start URLs; each one's host is in scope. For clean per-site reports, run one site per run (for example with an Apify task per client).

**How fresh is the data?**
Every run fetches the pages live, so results reflect the site at run time.

**Can you add a check I need?**
Yes. Open an issue with the check you want. If the actor saves you time, a rating on the Store page helps others find it.

### Related actors

- [Domain WHOIS & DNS API](https://apify.com/teddythebuilder/domain-whois-dns-api): bulk WHOIS/RDAP registrar and expiry dates, domain availability, DNS records, SPF/DMARC email security grade and SSL certificate expiry in one row per domain.
- [Tech Stack Detector](https://apify.com/teddythebuilder/tech-stack-detector): find the CMS, ecommerce platform, analytics and marketing tools a website uses. Pair it with this audit for a full website report.
- [Career Site Jobs API](https://apify.com/teddythebuilder/career-site-jobs-api): jobs straight from company career sites on Greenhouse, Lever, Ashby, Workable, Recruitee and Teamtailor.
- [Nordic Company Registry API](https://apify.com/teddythebuilder/nordic-company-registry): companies from Sweden, Norway and Finland with industry codes, for building lead lists to audit.
- [Swedish Jobs Scraper](https://apify.com/teddythebuilder/swedish-jobs-monitor) and [Nordic Tenders Scraper](https://apify.com/teddythebuilder/nordic-public-tenders-monitor).

### Changelog and support

- **1.0 (2026-10):** first release. Crawl, sitemap and list modes; 35 issue checks with severities and a 0–100 score; free internal and external link checking; sitemap coverage and orphan pages; duplicate title and description detection; site SUMMARY report.

Questions or bugs? Open an issue. We reply within 24 hours.

# Actor input Schema

## `startUrls` (type: `array`):

The site to audit (its homepage), e.g. "https://www.example.com/". In list mode, every URL you add is audited once.

## `maxPages` (type: `integer`):

Maximum number of pages to audit. Each audited page is one billable result, e.g. 500 for a small site.

## `crawlMode` (type: `string`):

crawl = follow internal links from the start URL and add sitemap URLs (finds orphan pages); sitemap = audit only the URLs listed in the sitemap; list = audit exactly the URLs you entered.

## `checkExternalLinks` (type: `boolean`):

Also check outgoing links to other websites (HEAD request, GET fallback). Internal links are always checked.

## `onlyPagesWithIssues` (type: `boolean`):

Skip pages with no issues at all. Skipped pages are not charged. The SUMMARY record always covers every page.

## `useSitemap` (type: `boolean`):

Read the sitemaps listed in robots.txt (or /sitemap.xml) to find orphan pages and flag indexable pages missing from the sitemap.

## `includeSubdomains` (type: `boolean`):

Treat subdomains (e.g. blog.example.com) as part of the site. www and non-www are always the same site.

## `includeUrlPatterns` (type: `array`):

Optional. Audit only URLs matching at least one regular expression, e.g. "/blog/".

## `excludeUrlPatterns` (type: `array`):

Optional. Skip URLs matching any regular expression, e.g. "/tag/" or "sort=".

## `maxDepth` (type: `integer`):

How many clicks away from the start URL to crawl. 0 = unlimited.

## `maxLinkChecks` (type: `integer`):

Maximum number of uncrawled link targets (external links, internal links beyond the page limit) to check for 404s. Link checks are free.

## `maxConcurrency` (type: `integer`):

Pages fetched in parallel. Keep it low (3-10) to be gentle on the website. robots.txt crawl-delay is always respected.

## `slowPageMs` (type: `integer`):

Pages whose server response takes longer than this are flagged slow_response.

## `timeoutSecs` (type: `integer`):

Per-page HTTP timeout.

## Actor input object example

```json
{
  "startUrls": [
    "https://crawler-test.com/"
  ],
  "maxPages": 20,
  "crawlMode": "crawl",
  "checkExternalLinks": true,
  "onlyPagesWithIssues": false,
  "useSitemap": true,
  "includeSubdomains": false,
  "maxDepth": 0,
  "maxLinkChecks": 100,
  "maxConcurrency": 5,
  "slowPageMs": 2000,
  "timeoutSecs": 20
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://crawler-test.com/"
    ],
    "maxPages": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("teddythebuilder/website-seo-audit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://crawler-test.com/"],
    "maxPages": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("teddythebuilder/website-seo-audit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://crawler-test.com/"
  ],
  "maxPages": 20
}' |
apify call teddythebuilder/website-seo-audit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,teddythebuilder/website-seo-audit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/683tZrEHasi4dhvNZ/builds/0ikfaxxSBKeq5X2NR/openapi.json