# SEO Audit & Broken Link Checker (`lintlab/seo-site-qa`) Actor

HTTP-only technical SEO audit and broken link checker: titles, meta, headings, canonicals, robots, hreflang, images, structured data, duplicates, and XML sitemaps. JSON/Markdown reports for CI and AI agents. No JS rendering.

- **URL**: https://apify.com/lintlab/seo-site-qa.md
- **Developed by:** [Lintlab](https://apify.com/lintlab) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$4.00 / 1,000 page auditeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Technical SEO Site Audit

Crawl public pages or XML sitemaps and get deterministic, actionable technical SEO findings for every page, plus concise JSON and Markdown site summaries.

### What it checks

- Titles, meta descriptions, headings, canonicals, robots directives, hreflang, images, and structured data
- Duplicate titles and descriptions across the audited site
- Internal links with cached HEAD checks and a GET fallback
- Ordinary, gzip-compressed, and nested sitemap indexes
- $0.004 per successfully fetched and saved page

This Actor is HTTP-only and does not render JavaScript.

### Quick start

```json
{"startUrls":[{"url":"https://example.com"}],"discoverSitemap":true,"maxPages":100,"checkLinks":true}
```

### Use with AI agents / MCP

Call `lintlab/seo-site-qa` through the Apify API or Apify MCP server. Use the per-page `findings` for precise fixes and `SUMMARY` or `SUMMARY.md` for a site-level plan.

### Overview

SEO Site QA is an HTTP-only technical SEO audit for sites you own or are authorized to audit. It accepts pages, an XML sitemap, or a sitemap index; checks each allowed page; and writes compact, deterministic JSON plus a site summary. It is intended for developers, agencies, CI workflows, and AI agents that need findings they can act on without running a browser.

### What it does

- Reads ordinary and gzip-compressed sitemaps, including nested sitemap indexes.
- Optionally discovers a sitemap from `robots.txt` and then `/sitemap.xml` when given only a homepage.
- Uses Crawlee's `CheerioCrawler` with guarded HTTP requests. It does not launch a browser.
- Audits up to 5,000 pages and can optionally discover more pages from internal links.
- Checks internal links with cached HEAD requests and a GET fallback, within a configurable cap.
- Produces one dataset item per attempted page, `SUMMARY` JSON, and `SUMMARY.md`.
- Charges only after an allowed, successfully fetched page has been saved to the dataset.

### Input

```json
{
  "startUrls": [{ "url": "https://crawlee.dev" }],
  "discoverSitemap": true,
  "maxPages": 100,
  "sameHostOnly": true,
  "followLinks": false,
  "checkLinks": true,
  "maxLinkChecks": 500,
  "timeoutSecs": 20,
  "concurrency": 5
}
```

You may use `sitemapUrl` instead of, or together with, `startUrls`. `startUrls` also accepts plain URL strings through the programmatic input path. Global concurrency defaults to 5; requests to one origin are limited to 2 concurrently.

### Page output

Each dataset item describes one attempted page. Blocked and failed pages include an `error` and are not charged.

```json
{
  "url": "https://example.com/about",
  "finalUrl": "https://example.com/about",
  "status": 200,
  "redirectChain": [],
  "title": "About Example Company",
  "metaDescription": "How our team builds useful products.",
  "h1": ["About us"],
  "canonical": "https://example.com/about",
  "indexable": true,
  "wordCount": 438,
  "jsonLdTypes": ["AboutPage"],
  "findings": [
    {
      "checkId": "opengraph.image_missing",
      "severity": "notice",
      "message": "Open Graph image is missing.",
      "fix": "Add a og:image meta tag for richer social previews."
    }
  ],
  "counts": { "error": 0, "warning": 0, "notice": 1 },
  "checkedAt": "2026-01-01T00:00:00.000Z"
}
```

`redirectChain` contains `{url, status, location}` objects. A finding always contains `checkId`, `severity`, `message`, and `fix`; `value` is included when useful.

### Checks

Check IDs are stable API values. New checks may be added in later versions.

| Check ID | Severity | Condition |
| --- | --- | --- |
| `title.missing` | error | No `<title>` element |
| `title.empty` | error | Title element has no text |
| `title.too_long` | warning | Title is over 60 characters |
| `title.too_short` | warning | Title is under 10 characters |
| `title.duplicate` | warning | Same non-empty title appears on multiple audited pages |
| `description.missing` | warning | Meta description is absent or empty |
| `description.too_long` | warning | Meta description is over 160 characters |
| `description.duplicate` | warning | Same non-empty description appears on multiple audited pages |
| `h1.missing` | error | No H1 heading |
| `h1.multiple` | warning | More than one H1 |
| `headings.level_skip` | warning | Heading order jumps by more than one level |
| `canonical.missing` | warning | Canonical link is absent or empty |
| `canonical.relative` | warning | Canonical is not written as an absolute URL |
| `canonical.elsewhere` | warning | Canonical resolves to a different URL |
| `canonical.malformed` | error | Canonical cannot be parsed as a URL |
| `canonical.non_200` | error | Canonical target does not return HTTP 200 or cannot be fetched |
| `robots.meta_noindex` | warning | Meta robots contains `noindex` |
| `robots.meta_nofollow` | warning | Meta robots contains `nofollow` |
| `robots.x_noindex` | warning | X-Robots-Tag contains `noindex` |
| `robots.x_nofollow` | warning | X-Robots-Tag contains `nofollow` |
| `robots.blocked_sitemap` | error | URL is in a sitemap but disallowed by robots.txt |
| `robots.disallowed` | error | Non-sitemap start/discovered URL is disallowed |
| `http.non_200` | error | Final page response is outside 2xx |
| `sitemap.redirected` | warning | A URL taken from the sitemap redirects |
| `html.lang_missing` | warning | `<html>` has no `lang` |
| `html.lang_malformed` | warning | `lang` is not a valid BCP 47 tag |
| `viewport.missing` | warning | Viewport meta tag is absent |
| `images.alt_missing` | warning | One or more images lack an `alt` attribute; value has count and first five sources |
| `opengraph.title_missing` | notice | `og:title` is absent or empty |
| `opengraph.description_missing` | notice | `og:description` is absent or empty |
| `opengraph.image_missing` | notice | `og:image` is absent or empty |
| `jsonld.invalid` | error | A JSON-LD block does not parse as JSON |
| `jsonld.types` | notice | Valid JSON-LD types were found; types are reported in `value` and `jsonLdTypes` |
| `resources.mixed_content` | error | An HTTPS page references an `http://` resource |
| `html.too_large` | warning | Uncompressed HTML is over 1.5 MB |
| `hreflang.malformed` | warning | Hreflang language or absolute HTTP(S) target is malformed |
| `links.broken_internal` | error | Internal link returns 4xx/5xx or a network error |
| `links.redirect_chain` | warning | Internal link has more than one redirect hop |
| `fetch.failed` | error | Page fetch fails before an HTTP response is available |

An empty JSON-LD set is not an error. If valid JSON-LD exists, its `@type` values are reported. Link findings are attached to each source page; the summary also lists broken source-to-target relationships.

### Summary and score

The default key-value store receives:

- `SUMMARY`: JSON with pages audited/attempted, aggregate counts, top issues, duplicate title and description groups, broken links, sitemap errors, and the score.
- `SUMMARY.md`: the same core results in a readable Markdown report.

The score is:

```text
100 - min(100, (errors × 5 + warnings × 2 + notices × 0.25) / successful pages)
```

It is clamped to 0–100 and rounded to one decimal. The formula is deliberately simple and comparative; a business-critical issue can matter more than its numeric weight.

### Pricing

The `page-audited` event costs **$0.004 per successfully audited page**, or **$4 per 1,000 pages**. Internal link checks are free and bounded by `maxLinkChecks`. Disallowed pages, network failures, and non-2xx final page responses are written for diagnosis but are not charged. The Actor stops writing further page results when the platform reports that the event charge limit has been reached.

### Limits and behavior

- Maximum 5,000 scheduled pages per run and 10,000 unique internal link checks.
- Maximum 10 redirects per HTTP operation.
- HTML responses are capped at 10 MB; sitemap responses at 20 MB.
- Sitemap indexes are capped at 100 fetched sitemap documents per run.
- Link checks use HEAD, falling back to GET for HTTP 405 or 501. Canonical targets are checked separately and cached.
- `sameHostOnly` compares hostnames. URL fragments are removed during normalization.
- The crawler evaluates server-delivered HTML only. It does not execute JavaScript, measure rendered layout, or run Core Web Vitals.
- Hreflang validation checks language syntax and absolute HTTP(S) targets; it does not verify reciprocal annotations.
- Duplicate matching is case-insensitive after trimming, not semantic.
- `robots.txt` rules support user-agent groups, Allow/Disallow, `*`, and terminal `$`; unusual nonstandard directives are ignored.

### Safety and compliance

Use this Actor only for sites you own or have permission to audit.

- User agent: `lintlab-seo-qa/1.0 (+https://github.com/lintlab)`; robots product token: `lintlab-seo-qa`.
- `robots.txt` is cached per origin. Fetch/network errors and 5xx responses fail closed (all URLs on that origin are disallowed); 4xx responses allow crawling under RFC 9309 behavior.
- Every page, sitemap, canonical, link check, and redirect hop is restricted to HTTP(S), DNS-resolved before connecting, and blocked if any result is private, loopback, link-local, CGNAT, multicast, reserved, unspecified, or IPv6 ULA/IPv4-mapped private space.
- Redirects are followed manually, at most 10 hops, so every destination receives the same SSRF validation.
- No proxy, login/cookie input, CAPTCHA handling, browser, or stealth behavior is used.
- The output does not extract emails, phone numbers, or personal names.

### Local development

Requires Node.js 20 or newer.

```sh
npm ci
npm test
APIFY_LOCAL_STORAGE_DIR=./storage CRAWLEE_STORAGE_DIR=./storage node src/main.js
```

Before the final command, write the desired Actor input to `storage/key_value_stores/default/INPUT.json` (or use `apify run --purge` with the Apify CLI).

***

Built by **lintlab** — small, reliable data tools. AI-assisted (Claude/Codex), reviewed before release.
Support: lintlab.dev@gmail.com · https://github.com/lintlab

# Actor input Schema

## `startUrls` (type: `array`):

Pages to audit. With one homepage and sitemap discovery enabled, the Actor also checks robots.txt and /sitemap.xml.

## `sitemapUrl` (type: `string`):

Optional XML sitemap or sitemap index URL. Gzip-compressed sitemaps are supported.

## `discoverSitemap` (type: `boolean`):

When the only start URL is a homepage, try Sitemap directives in robots.txt and then /sitemap.xml.

## `maxPages` (type: `integer`):

Maximum number of unique pages scheduled for audit.

## `sameHostOnly` (type: `boolean`):

Ignore sitemap entries and followed links on other hostnames.

## `followLinks` (type: `boolean`):

Add internal links found on audited pages to the audit queue, bounded by maximum pages.

## `checkLinks` (type: `boolean`):

Check internal link targets with HEAD and a GET fallback. Link checks are not charged.

## `maxLinkChecks` (type: `integer`):

Maximum unique internal link targets checked across the run.

## `timeoutSecs` (type: `integer`):

Timeout applied to each HTTP request.

## `concurrency` (type: `integer`):

Maximum concurrent page tasks. Requests to any one host are limited to two at a time.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ],
  "discoverSitemap": true,
  "maxPages": 100,
  "sameHostOnly": true,
  "followLinks": false,
  "checkLinks": true,
  "maxLinkChecks": 500,
  "timeoutSecs": 20,
  "concurrency": 5
}
```

# Actor output Schema

## `pages` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `summaryMarkdown` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://crawlee.dev"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("lintlab/seo-site-qa").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://crawlee.dev" }] }

# Run the Actor and wait for it to finish
run = client.actor("lintlab/seo-site-qa").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://crawlee.dev"
    }
  ]
}' |
apify call lintlab/seo-site-qa --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,lintlab/seo-site-qa"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/SwggFLZIVFbAwipoj/builds/r3ODUf2X9nbV4fje5/openapi.json