# SEO Audit Crawler — Technical SEO Site Audit (`nexgensignal/seo-audit-crawler`) Actor

Crawl a website and audit every page: status codes, redirects, titles, meta descriptions, H1s, canonicals, noindex, hreflang, schema, missing alt text, broken pages and links. One row per page with its issues, plus a site summary. Part of NexGen Signal — data tools and official-source data products

- **URL**: https://apify.com/nexgensignal/seo-audit-crawler.md
- **Developed by:** [NexGen Signal](https://apify.com/nexgensignal) (community)
- **Categories:** SEO tools, Automation, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $33.50 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🧭 SEO Audit Crawler — Technical SEO Site Audit

Crawl a website and get **a technical SEO audit of every page**: status code and redirect chain, title and meta description with lengths, H1s and H2 count, canonical (and whether it points to the page itself), meta robots and X-Robots-Tag with an indexable flag, lang, viewport, Open Graph and Twitter tags, hreflang, schema.org types, images without alt, internal / external / nofollow links, word count, page size and download time.

Each page row lists its issues in plain words (missing title, title over 60 characters, missing meta description, more than one H1, noindex, canonical points elsewhere, thin content, images without alt, slow response, broken external links when you turn that check on). A `SITE_REPORT` record sums the issues across the site and lists broken internal pages with the pages that link to them, and duplicate titles and descriptions.

The unit of output is one page audited, including 4xx and 5xx pages. Links that are skipped (robots.txt, not HTML) are not billed.

### Who buys this and for what job

- **SEO consultants and agencies** auditing client sites and tracking fixes run after run.
- **Site owners and developers** finding broken pages, redirect chains and missing tags before a launch.
- **Content teams** spotting thin pages and duplicate titles.
- **Analysts** feeding page-level SEO data into dashboards.

Buyers usually search for this as **"SEO audit"**, **"site audit crawler"** or **"broken link checker"**.

### 📊 Sample Output

![Table of audited pages: URL, status code, title length, meta description length, H1 count, word count and issues.](https://api.apify.com/v2/key-value-stores/IXCaMKjxSmUTLHhmq/records/seo-audit-crawler.png)

| url | status_code | title_length | meta_description_length | h1_count | issues |
|---|---|---|---|---|---|
| https://www.federalreserve.gov/ | 200 | 28 | 56 | 1 | no canonical |
| https://www.federalreserve.gov/default.htm | 200 | 28 | 56 | 1 | no canonical |
| https://www.federalreserve.gov/feeds/feeds.htm | 200 | 33 | 56 | 1 | no canonical |
| https://www.federalreserve.gov/newsevents/calendar.htm | 200 | 32 | 56 | 1 | no canonical |
| https://www.federalreserve.gov/subscribe.htm | 200 | 57 | 56 | 1 | no canonical |

Rows from live run `jyZFG1qNTMzPxp2so` on Apify (40 pages of federalreserve.gov). Notice the most common issue: the pages carry no canonical tag.

### ✅ What you get

Every field of each record:

- `record_id`, `url`, `final_url`, `status_code`, `redirect_chain`, `depth`, `found_on` — the page, its answer and redirects, how deep it is and where it was found.
- `content_type`, `bytes`, `load_ms` — type, HTML size and download time.
- `title`, `title_length`, `meta_description`, `meta_description_length` — the snippet tags.
- `meta_robots`, `x_robots_tag`, `indexable`, `canonical`, `canonical_is_self` — indexing signals.
- `lang`, `has_viewport`, `h1`, `h1_count`, `h2_count`, `word_count` — page basics.
- `og_title`, `og_description`, `og_image`, `twitter_card`, `hreflang`, `structured_data_types` — social, language and schema tags.
- `images_total`, `images_missing_alt`, `internal_links`, `external_links`, `nofollow_links`, `broken_external_links` — images and links.
- `issues`, `issue_count`, `crawled_at` — issues found on the page and when it was crawled (UTC).

### ⚙️ Sample inputs

**A 10-page sample audit (the Store example)**

```json
{
  "startUrls": ["https://www.federalreserve.gov/"],
  "maxPages": 10
}
```

Start page plus pages it links to and the sitemap lists.

**One section only, with broken-link check**

```json
{
  "startUrls": ["https://www.federalreserve.gov/"],
  "includeUrlPatterns": ["/newsevents/"],
  "checkExternalLinks": true,
  "maxPages": 200
}
```

Crawl the news section and check its external links.

**Full site including subdomains**

```json
{
  "startUrls": ["https://www.example.com/"],
  "includeSubdomains": true,
  "excludeUrlPatterns": ["\\?page=", "/search"],
  "maxPages": 5000
}
```

Skips paginated and search URLs.

Input fields:

| Field | Meaning |
|---|---|
| `startUrls` | sites to audit (up to 100); the crawl stays on each site. |
| `maxPages` | stop after this many audited pages (default 100). Each audited page is billed. |
| `maxDepth` | clicks from the start page (default 10). |
| `useSitemaps` | also queue sitemap pages (default true). |
| `includeSubdomains` | crawl subdomains too (default false). |
| `includeUrlPatterns`, `excludeUrlPatterns` | regular expressions for URLs to crawl or skip. |
| `checkExternalLinks` | test external links once each (default false). |

### 🧾 JSON sample record

```json
{
  "record_id": "f3d937f613f2f6d80e4375fc",
  "url": "https://www.federalreserve.gov/",
  "final_url": "https://www.federalreserve.gov/",
  "status_code": 200,
  "redirect_chain": null,
  "depth": 0,
  "found_on": null,
  "content_type": "text/html",
  "bytes": 106057,
  "load_ms": 30,
  "title": "Federal Reserve Board - Home",
  "title_length": 28,
  "meta_description": "The Federal Reserve Board of Governors in Washington DC.",
  "meta_description_length": 56,
  "meta_robots": null,
  "x_robots_tag": null,
  "indexable": true,
  "canonical": null,
  "canonical_is_self": null,
  "lang": "en",
  "has_viewport": true,
  "h1": [
    "Board of Governors of the Federal Reserve System"
  ],
  "h1_count": 1,
  "h2_count": 2,
  "word_count": 1816,
  "og_title": "Home",
  "og_description": "The Federal Reserve Board of Governors in Washington DC.",
  "og_image": "https://www.federalreserve.gov/images/social-media/social-default-image-opengraph.jpg",
  "twitter_card": "summary",
  "hreflang": null,
  "structured_data_types": null,
  "images_total": 9,
  "images_missing_alt": 0,
  "internal_links": 212,
  "external_links": 19,
  "nofollow_links": 0,
  "broken_external_links": null,
  "issues": "no canonical",
  "issue_count": 1,
  "crawled_at": "2026-10-17T03:55:10Z"
}
```

### 🔧 How it works

The crawl runs 5 pages at a time, breadth first, and honours a site's robots.txt crawl-delay. Files such as images, PDFs and scripts are not crawled; pages that are not HTML are counted and not billed. www and non-www count as the same site.

Every run reads each host's robots.txt first: a path it disallows is skipped, listed in the run receipt and never charged. Only public internet addresses are fetched (no private, loopback or cloud-metadata addresses, checked again on every redirect). No logins, no CAPTCHA solving, no proxies.

Delivery is pushed first and charged after. `SITE_REPORT` holds the site summary; `RUN_RECEIPT` lists each start URL's outcome, page outcomes with counts, the robots.txt and sitemap results, and the delivered and charged counts.

### 💰 Pricing example

Per page delivered, no start fee, no monthly charge:

| Apify plan | Price per page |
|---|---|
| Free | $0.05 |
| Bronze | $0.045 |
| Silver | $0.04 |
| Gold, Platinum, Diamond | $0.0335 |

- 10 pages: $0.50 on the Free plan, $0.335 on Gold.
- 500 pages: $25.00 on Free, $16.75 on Gold.
- 5,000 pages: $250.00 on Free, $167.50 on Gold.

**Coverage and size.** Proof runs on 17 Oct 2026: 40 federalreserve.gov pages audited in about 7 seconds with 759 more pages queued; 8 NASA pages with their external links checked in about 10 seconds. Pages built only by JavaScript are audited as the server sends them.

### ⚖️ Legal & ToS

Audit sites you own or are allowed to crawl. Every run reads each host's robots.txt first: a path it disallows is skipped, listed in the run receipt and never charged. Only public internet addresses are fetched (no private, loopback or cloud-metadata addresses, checked again on every redirect). No logins, no CAPTCHA solving, no proxies.

### ❓ FAQ

**Does it render JavaScript?**
No. It audits the HTML the server sends, as most search-engine first passes do.

**Is a 404 page billed?**
Yes: it is an audited page, and it is listed in SITE_REPORT with the pages that link to it.

**Can I audit only some pages?**
Yes, with includeUrlPatterns, excludeUrlPatterns, maxDepth and maxPages.

### 🆘 Troubleshooting

- **Only a few pages audited** — Check maxPages, maxDepth and your patterns; pages linked only by JavaScript are not found.
- **No pages audited** — The start URL was blocked by robots.txt or was not HTML; see RUN_RECEIPT.
- **SOURCE UNAVAILABLE** — The start URLs could not be reached after retries. Nothing was charged.

### 🔗 Related Actors

- [Website Performance Audit](https://apify.com/nexgensignal/website-performance-audit-api) — Lighthouse scores and web vitals per page
- [Website Tech Stack Detector](https://apify.com/nexgensignal/website-tech-stack-api) — what a site is built with
- [Website Contact Scraper](https://apify.com/nexgensignal/bulk-website-contact-extractor) — contact details from website lists

🏢 **About NexGen Signal** — data tools and official-source data products on Apify. Browse the catalog: [apify.com/nexgensignal](https://apify.com/nexgensignal)

### ⭐ Found this useful?

If this actor saved you time, a rating on its Apify Store page helps other buyers find it.

Part of NexGen Signal — data tools and official-source data products.

# Actor input Schema

## `startUrls` (type: `array`):

Start pages; the crawl stays on each site (www and non-www count as one). Up to 100.

## `maxPages` (type: `integer`):

Stop after this many pages are audited. Each audited page (including 4xx/5xx pages) is billed; skipped and non-HTML links are not.

## `maxDepth` (type: `integer`):

How many clicks from the start page to follow (0 = start pages only).

## `useSitemaps` (type: `boolean`):

Also queue pages listed in the site's sitemap (from robots.txt, or /sitemap.xml).

## `includeSubdomains` (type: `boolean`):

Also crawl subdomains, e.g. blog.example.com.

## `includeUrlPatterns` (type: `array`):

Regular expressions; when set, only matching URLs are crawled (start pages always are).

## `excludeUrlPatterns` (type: `array`):

Regular expressions for URLs not to crawl, e.g. /search or ?page=

## `checkExternalLinks` (type: `boolean`):

Request each external link once (up to 50 per page, 2,000 per run) and report broken ones. Slower.

## Actor input object example

```json
{
  "startUrls": [
    "https://www.federalreserve.gov/"
  ],
  "maxPages": 10,
  "maxDepth": 10,
  "useSitemaps": true,
  "includeSubdomains": false,
  "includeUrlPatterns": [],
  "excludeUrlPatterns": [],
  "checkExternalLinks": false
}
```

# Actor output Schema

## `results` (type: `string`):

The delivered (and billed) records.

## `receipt` (type: `string`):

Per-link outcome, robots result, delivered and charged counts.

## `site_report` (type: `string`):

Site summary: issues, broken pages with the pages linking to them, duplicate titles and descriptions.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://www.federalreserve.gov/"
    ],
    "maxPages": 10
};

// Run the Actor and wait for it to finish
const run = await client.actor("nexgensignal/seo-audit-crawler").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://www.federalreserve.gov/"],
    "maxPages": 10,
}

# Run the Actor and wait for it to finish
run = client.actor("nexgensignal/seo-audit-crawler").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://www.federalreserve.gov/"
  ],
  "maxPages": 10
}' |
apify call nexgensignal/seo-audit-crawler --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nexgensignal/seo-audit-crawler"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9y9UdwRVKJNa3NOAW/builds/jF4YVCENrzJH9EWDL/openapi.json