# SEO Audit Crawler - Broken Links, Meta Tags, Canonicals (`sipki/seo-audit-http`) Actor

Crawl a website and get a technical SEO audit: per-page status, titles and meta descriptions, H1, canonicals, noindex, hreflang, schema.org, image alt and broken links, plus a site summary with a 0-100 score, prioritised issues and sitemap check. $10 per 1,000 pages.

- **URL**: https://apify.com/sipki/seo-audit-http.md
- **Developed by:** [SIPKI LLC](https://apify.com/sipki) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$10.00 / 1,000 audited pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SEO Audit Crawler - broken links, meta tags, canonicals and a 0-100 site score

**Give it a start URL; get one record per page plus a site summary with a 0-100 score and issues ranked by importance - $10 per 1,000 pages.**

Finds broken internal and external links, missing or duplicate titles and meta descriptions, missing H1, canonicals pointing elsewhere, noindex pages, hreflang and schema.org gaps, images without alt, slow pages, and sitemap.xml mismatches. Plain HTTP, no browser: fast and cheap for sites that render on the server.

**Try it now:** press **Start** with the prefilled input (10 pages of docs.apify.com). The audit costs $0.10.

### What you get

**Per page:** HTTP status, redirect chain, response time, HTML size, crawl depth, title and meta description (text, length, duplicates), H1 (count), canonical (and whether it points elsewhere), robots meta / X-Robots-Tag (indexable or not), hreflang, Open Graph and Twitter tags, schema.org types (JSON-LD and microdata), images without `alt`, internal and external link counts, and **broken internal / external links** found on that page, plus a list of issues with severity (critical, high, medium, low).

**Per site:** score, issue counts, issues ranked by severity and by number of pages affected, broken links list, **sitemap.xml comparison** (crawled pages missing from the sitemap, sitemap URLs that return errors, URLs not reached), **llms.txt** presence, robots.txt state, average response time, pages skipped because robots.txt forbids them.

### How it behaves (read before relying on it)

- Plain HTTP only: **no browser, no JavaScript rendering, no proxies, no CAPTCHA solving, no attempt to get around blocks.** Pages that are built only in the browser, Core Web Vitals and rendering checks are **not** covered.
- **robots.txt is always honoured** (RFC 9309), on every page, every redirect hop and also for the HEAD checks of external links. Disallowed pages are not fetched, not counted and not charged. If robots.txt cannot be read (5xx, 401/403/429 or network error) the site is not crawled. `Crawl-delay` is respected (up to 10 s).
- **Polite:** by default 1 request per second per site within a run, never more than 2 (configurable lower). `www.example.org` and `example.org` share one limit. Only one crawl of a given site runs at a time in a run. Honest user agent `SipkiSEOAudit/1.0 (+https://apify.com/sipki/seo-audit-http)`, never changed or rotated.
- **When a site says slow down or no:** on HTTP 429 or 503 the Actor pauses for the `Retry-After` time (at least 10 seconds) and retries the same request; after 3 such answers in a row it stops that site (`stopped_by: rate_limited` in the site summary). On 403/429 for a page or link it records the status and does not retry with anything else: no proxy, no identity change, no workaround.
- **Internal addresses are refused:** start URLs, redirects, sitemap and link targets that lead to private, loopback, link-local or cloud-metadata addresses are never contacted.
- Stays on the start host (www and non-www are one site). Pages are fetched breadth-first from the start URL; if the link graph runs out before `maxPages`, URLs from the sitemap are used.
- Link checks use HEAD (and one GET when a server answers HEAD with 400/404/405/501; not after 403). Answers 401/403/429/451/999 are counted as "restricted for bots", **not** as broken. Check budget per site is `maxLinkChecks` (about one third reserved for external links); the rest is reported as not checked.
- Use it on sites you own or are allowed to audit. You are responsible for complying with the law and the terms of the sites you crawl.
- A start URL that redirects to another host is reported and not followed.

### Input

| Field | Meaning |
|---|---|
| `startUrls` | URLs or bare domains, one per line; each distinct site is crawled separately |
| `maxPages` | Pages per site (default 10, max 2000) - also your cost cap |
| `checkExternalLinks` | HEAD-check external links (default on) |
| `maxLinkChecks` | Link-check budget per site (default 20) |
| `checkSitemap` | Compare with sitemap.xml and check llms.txt (default on) |
| `requestsPerSecond` | Per site, 0.2 to 2 (default 1) |

### Output

Dataset with two record types: `type: "page"` and `type: "site"` (use the "Pages" and "Site summaries" views). A run summary is in the key-value store (`RUN_SUMMARY`).

Score: starts at 100; every issue type subtracts its weight (critical 15, high 8, medium 4, low 1.5) scaled by the share of pages it affects, with a floor of 30 % of the weight if the issue exists at all. Site-level issues (no sitemap, start URL error) subtract the full weight. It is a prioritisation aid, not a ranking factor of any search engine.

### Pricing

Pay per event: **$0.01 per page** (event `page`). Free: pages that failed at network level or were skipped, answers 429, redirects that lead nowhere, the site summary. Your spending limit is respected: the run stops when it is reached.

### Example (our own test, 10 public sites, 50 pages each)

451 pages in 325 s locally (4 sites in parallel, 2 requests/s per site; measured before the default was lowered to 1 request/s, so expect about twice as long now). Typical findings: duplicate titles, missing canonicals, missing structured data, a handful of broken internal links.

### How to use

1. Paste one or more start URLs or bare domains into **Start URLs**.
2. Set **Max pages per site** - it is also your cost cap (10 pages = $0.10, 500 pages = $5.00).
3. Press **Start**. Open the "Pages" and "Site summaries" views, export as JSON, CSV or Excel, or call it from the Apify API, a schedule or the Apify MCP server.

### Who uses it

- **SEO freelancers and agencies** - audit client sites and get the findings as data, not a PDF.
- **Site owners and developers** - schedule a weekly check for broken links and noindex mistakes.
- **Content teams** - find duplicate titles and missing descriptions across many pages.

### More questions

**Does it check Core Web Vitals or JavaScript-rendered pages?** No. It works over plain HTTP, so pages built only in the browser are not covered.
**Are 403 answers counted as broken links?** No. 401/403/429/451/999 are reported as "restricted for bots", not broken.
**How much does a 500-page audit cost?** $5.00. Pages skipped by robots.txt and network failures are free.

### Not affiliated

Not affiliated with, endorsed or sponsored by Google, Bing or any other search engine or website it audits. The score is a prioritisation aid, not a ranking factor of any search engine.

### Content rights

The Actor stores only short technical fragments of pages you audit (title, meta description, H1, counts, status codes), not page content. You are responsible for auditing only sites you own or are allowed to audit.

### Acceptable use

This Actor is a tool. You decide which sites, companies or files it processes and what you do with the results, and you are responsible for that use. Use it only where you have the right to access the data, and in line with applicable law (including data-protection, anti-spam and copyright law) and the terms of the sources. Do not use it to send unsolicited messages, to build lists for spam, to bypass blocks or access controls, or to place excessive load on any site. If a site owner asks us to stop, we may exclude that site. Sipki, LLC does not receive or store your results; they stay in your Apify storage.

### No warranty

The Actor and its output are provided "as is", without warranty of accuracy, completeness or fitness for a purpose. Verify results before relying on them. Liability of Sipki, LLC is limited as set out in the Apify Standard Actor Contract that governs your use of this Actor.

### Personal data

Results may contain personal data (for example, a name in a page title or an e-mail address in a link). For such data you are the controller and Sipki, LLC acts only as your processor under section 5 of the Apify Standard Actor Contract. You need a lawful basis for your processing and must handle data-subject requests. To request deletion of anything related to you, open an Issue on this Actor's page.

### For site owners (opt-out)

The crawler identifies itself as `SipkiSEOAudit/1.0 (+https://apify.com/sipki/seo-audit-http)`, follows robots.txt and crawl-delay and limits itself to 1 request per second per site within a run by default. To have your domain excluded, or to report a problem, open an Issue on this Actor's page: your domain will be added to the block list within 3 business days, and every run checks that list.

# Actor input Schema

## `startUrls` (type: `array`):

One per line. Each distinct site (host) is crawled separately by internal links, staying on that host (www and non-www are treated as the same site, with one shared rate limit).

## `maxPages` (type: `integer`):

Hard cap of crawled HTML pages per site. You are charged per page, so this also caps cost.

## `checkExternalLinks` (type: `boolean`):

Sends one HEAD request per distinct external link (robots.txt of the target host is respected). Turn off for faster runs.

## `maxLinkChecks` (type: `integer`):

Cap on HEAD requests for links that were not crawled (internal and external).

## `checkSitemap` (type: `boolean`):

Reads sitemap(s) listed in robots.txt or /sitemap.xml, reports crawled pages missing from the sitemap and sitemap URLs that were not reached. Also checks /llms.txt.

## `requestsPerSecond` (type: `number`):

Politeness limit per site (www and the bare domain share it; default 1, never above 2). A Crawl-delay in robots.txt slows it further. On HTTP 429/503 the Actor pauses (Retry-After, at least 10 s) and stops the site after 3 in a row.

## Actor input object example

```json
{
  "startUrls": [
    "https://docs.apify.com"
  ],
  "maxPages": 10,
  "checkExternalLinks": true,
  "maxLinkChecks": 20,
  "checkSitemap": true,
  "requestsPerSecond": 1
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://docs.apify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("sipki/seo-audit-http").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": ["https://docs.apify.com"] }

# Run the Actor and wait for it to finish
run = client.actor("sipki/seo-audit-http").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://docs.apify.com"
  ]
}' |
apify call sipki/seo-audit-http --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,sipki/seo-audit-http"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/amhgselkv6aBaJg3u/builds/OBvBMoYvgpIxigTtz/openapi.json