SEO Metadata Auditor for Crawled Datasets

Audit the crawl you already have. Find missing, duplicate and too-long titles and meta descriptions, H1 problems, canonical chains, thin content and messy URLs, with a site score and a reason for every point lost.

You ran Website Content Crawler (or any crawler) once. This Actor reads that dataset and runs an SEO metadata audit on it. It never fetches your site, never crawls and calls no external API, so there is nothing to block, nothing to wait for and no second crawl to pay for. Every issue says what was found, why it matters and how to fix it, and every score lists the exact points it lost.

What it checks

• Titles: missing, empty, too short, too long (characters and an estimated pixel width), duplicated across pages, placeholder or domain-only titles, keyword stuffing, identical to the H1.
• Meta descriptions: missing, empty, too short or long, duplicated, identical to the title, boilerplate.
• Headings: no H1, several H1s, empty or very long H1, skipped levels (h1 to h3), the same H1 on several pages.
• Canonicals: missing, invalid, pointing elsewhere, pointing at a page that is not a live indexable 200, chains and loops, another host, scheme mismatch, slash/case/www differences.
• Indexability and status: noindex, nofollow, noindex combined with a canonical elsewhere, HTTP 4xx/5xx and redirects found in the crawl.
• Content: thin pages and pages with identical text.
• URLs: long URLs, uppercase, underscores, tracking parameters, http pages on an https site, URL variants of the same page, pages too deep.
• Social, language, images: Open Graph and Twitter card tags, lang attribute, images without alt text.

The Actor looks across the whole dataset, so it finds problems a one-page checker cannot: duplicate titles, descriptions, H1s and content, URL variants and canonical chains.

It only checks what your dataset can prove

If the dataset has no heading data, the H1 checks are skipped and listed, not flagged on every page. The same goes for status codes, Open Graph tags, images and the rest. The run summary lists every skipped check and why, and each audited page repeats it in skippedChecks. That keeps the results trustworthy on crawler output that lacks some fields.

How to use it

1. Run your crawler. With Website Content Crawler, switch on Save Markdown so headings (H1 checks) and word counts are available. Copy the dataset ID of the run.
2. Run this Actor. Set Crawl dataset (or paste a few Pages), pick a Strictness profile and click Start.
3. Read the results. Open the dataset for one record per page, the Report for a client-ready summary, or download the Issue list CSV for a spreadsheet.

Chain it after Website Content Crawler

JavaScript (apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

// 1. Crawl the site (replace the input with your own; Save Markdown gives the H1 checks their data)
const crawl = await client.actor('apify/website-content-crawler').call({
  startUrls: [{ url: 'https://example.com' }],
  maxCrawlPages: 200,
  saveMarkdown: true,
});

// 2. Audit the crawl without crawling again
const audit = await client.actor('hiredhandai/seo-metadata-auditor').call({
  datasetId: crawl.defaultDatasetId,
  profile: 'standard',
});

// 3. Read the results
const { items } = await client.dataset(audit.defaultDatasetId).listItems();
console.log(items.filter((page) => page.score < 70).map((page) => page.url));
const summary = await client.keyValueStore(audit.defaultKeyValueStoreId).getRecord('OUTPUT');
console.log(summary.value.siteScore, summary.value.topIssues.slice(0, 3));

Python (apify-client):

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

audit = client.actor("hiredhandai/seo-metadata-auditor").call(
    run_input={"datasetId": "YOUR_CRAWL_DATASET_ID", "profile": "standard"}
)
store = client.key_value_store(audit["defaultKeyValueStoreId"])
print(store.get_record("OUTPUT")["value"]["siteScore"])
for page in client.dataset(audit["defaultDatasetId"]).iterate_items():
    if page["score"] < 70:
        print(page["url"], page["scoreReasons"])

It also works as a step in Make, n8n, Zapier and Google Sheets flows: crawl, then pass the crawl's defaultDatasetId to this Actor with your tool's Apify integration, and post the top issues to Slack, email or a sheet.

Other crawlers

Field names are detected automatically. The common shapes it understands, first match wins:

Fact	Looked up in
URL	url, crawl.loadedUrl, loadedUrl, finalUrl, pageUrl, link, address
Title	metadata.title, title, pageTitle, meta.title
Description	metadata.description, description, metaDescription, meta.description
Canonical	metadata.canonicalUrl, canonicalUrl, canonical, canonical_url
HTTP status	crawl.httpStatusCode, httpStatusCode, statusCode, status, response.status
Language	metadata.languageCode, lang, language, htmlLang
Robots	metadata.robots, robots, metaRobots, noindex, x-robots-tag header
Headings	headings, else markdown (ATX # lines), else html, else h1 / h1s
Page text	text, markdown, body, or wordCount

Anything else: use Field mapping, for example {"title": "meta.pageTitle", "url": "link"}.

For Cheerio Scraper or Web Scraper, return the fields this Actor reads from your pageFunction:

async function pageFunction(context) {
  const { $, request } = context;
  return {
    url: request.loadedUrl,
    title: $('title').first().text(),
    description: $('meta[name="description"]').attr('content') || null,
    canonical: $('link[rel="canonical"]').attr('href') || null,
    h1: $('h1').map((i, el) => $(el).text()).get(),
    lang: $('html').attr('lang') || null,
    robots: $('meta[name="robots"]').attr('content') || null,
    wordCount: $('body').text().split(/\s+/).filter(Boolean).length,
  };
}

Input

Field	Type	Default	Description
datasetId	string		The crawl dataset to audit. Pick it in the Console, or pass its ID through the API. Use this or pages.
pages	array		Up to 5,000 inline pages, to try the Actor or for small API jobs.
fieldMap	object	auto-detect	Map an audit field to your field name or dot path, e.g. {"title": "meta.pageTitle"}. Keys: url, title, description, canonical, statusCode, h1, headings, lang, robots, text, markdown, html, openGraph, twitter, images, depth, wordCount.
siteUrl	string	most common host	Main site address, used to spot canonicals that point to another host.
profile	standard / strict / lenient	standard	Threshold set for lengths, thin content, URL length and depth (table below).
rulesDisabled	array	[]	Rule codes to ignore, e.g. ["O005", "U006"]. Disabled rules do not affect the score.
crawlIsComplete	boolean	false	Turn on only if the dataset holds every page of the site. Then a canonical pointing to a page missing from the dataset (C008) is reported.
onlyStatusCodes	string	200	Statuses that get the full metadata checks, e.g. 200,2xx,301. Other pages only get a status issue. Pages without a status count as 200.
includeUrlPatterns	array	[]	Regular expressions; audit only URLs matching one of them.
excludeUrlPatterns	array	[]	Regular expressions; skip URLs matching any of them. Skipped pages are not audited and not charged.
analyzeContent	boolean	true	Word counts, thin content and identical-text detection. Turn off for a lighter run.
outputIssueFile	boolean	true	Also write every issue to ISSUES_CSV.
maxPages	integer	100000	Upper limit on pages read and audited (and charged).

URL patterns are limited to 50 per list and 200 characters each, nested quantifiers such as (a+)+ are rejected, and patterns are matched against the first 2,048 characters of the URL.

Profiles

Setting	Standard	Strict	Lenient
Title length (characters)	30 to 60	40 to 55	20 to 70
Title width (estimated pixels)	580	540	620
Description length	70 to 160	100 to 155	50 to 175
Thin content (warning / error, words)	under 200 / under 50	under 300 / under 100	under 150 / under 30
URL length	115	100	150
Query parameters	3	2	4
Crawl depth	4	3	6
Other		O005 and T008 become warnings	T008 off; H002, H006, U009 become info

A title counts as too long when its estimated width is over the pixel limit or it is more than 10 characters over the character limit. Pixel widths are estimates for an Arial-like 20 px font, not a measurement of what any search engine shows.

Output

One item per audited page in the default dataset. Real output from the prefilled example input (flat field names, no Open Graph or status data, so those checks are listed as skipped):

{
  "url": "https://example.com/products/widget",
  "normalizedUrl": "https://example.com/products/widget",
  "statusCode": null,
  "score": 85,
  "grade": "B",
  "indexable": true,
  "title": "Handmade Widget for Home and Office Use",
  "titleLength": 39,
  "titlePxEstimate": 381,
  "description": null,
  "descriptionLength": 0,
  "canonical": "https://example.com/products/widget",
  "h1": ["Handmade Widget"],
  "wordCount": 310,
  "issueCounts": { "error": 1, "warning": 0, "info": 0 },
  "issues": [
    {
      "code": "D001",
      "category": "description",
      "severity": "error",
      "message": "Meta description is missing.",
      "evidence": {},
      "fix": "Add a unique 70-160 character meta description that summarizes the page.",
      "penalty": 15
    }
  ],
  "scoreReasons": ["-15 D001 meta description missing"],
  "skippedChecks": [
    "H004, H007: only H1 text is available, not the heading outline",
    "I001-I003: no robots (noindex/nofollow) data in the dataset",
    "M001: no image data in the dataset",
    "O001-O005: no Open Graph data in the dataset",
    "S001, S002: no status code field in the dataset",
    "U009: no crawl depth in the dataset"
  ],
  "sourceIndex": 1
}

Each run also saves three records in the key-value store:

• OUTPUT: the site summary as JSON: siteScore and siteGrade, issue totals, topIssues, categoryScores, duplicateGroups (title, description, H1, content, URL variants), worstPages, fieldCoverage, rulesSkipped, and the run's recordsPushed and notPushedDueToSpendingLimit.
• REPORT_MD: a client-ready Markdown report built from the same data. Text that comes from your pages is escaped.
• ISSUES_CSV: url, code, severity, category, message, fix, one row per issue, safe to open in a spreadsheet (cells that start with =, +, - or @ are neutralized).

How scoring works

Every page starts at 100. Each issue subtracts 15 (error), 6 (warning) or 1 (info), and each category has a cap so one problem area cannot sink a page by itself: title 35, description 30, headings 25, canonical 30, indexability 40, content 20, URL 15, social/language/images 10, status 60. The score never goes below 0, grades are A (90+), B (80+), C (65+), D (50+), F, and scoreReasons lists every deduction, including the cap lines, so the numbers always add up.

The site score is the mean page score rounded, minus a duplicate-title penalty: 2 points for every 5% of pages in duplicate-title groups beyond the first 5%, at most 10. Only indexable pages that are not canonicalized elsewhere and have a checked status take part in duplicate groups, so noindex, canonicalized and error pages do not cause false duplicates. Pages marked noindex get title, description and heading findings reduced to info, because those are not shown in search results.

Rules version 2026.10.1. The thresholds are guidance, not search engine rules.

All rules

Code	Severity	What it flags	Needs in the dataset
S001	error	HTTP error status	HTTP status
S002	warning	Redirecting URL in crawl	HTTP status
I001	info	Noindex page	robots / noindex
I002	warning	Noindex with canonical elsewhere	robots / noindex
I003	info	Nofollow page	robots / noindex
T001	error	Title missing	title
T002	error	Title empty	title
T003	warning	Title short	title
T004	warning	Title long	title
T005	warning	Duplicate title	title
T006	warning	Generic title	title
T007	warning	Keyword stuffing in title	title
T008	info	Title equals H1	headings or H1
T009	info	Duplicate title apart from brand suffix	title
D001	error	Meta description missing	meta description
D002	error	Meta description empty	meta description
D003	warning	Meta description short	meta description
D004	warning	Meta description long	meta description
D005	warning	Duplicate meta description	meta description
D006	warning	Description equals title	meta description
D007	info	Boilerplate description	meta description
H001	error	No H1	headings or H1
H002	warning	Multiple H1	headings or H1
H003	error	Empty H1	headings or H1
H004	warning	Heading level skipped	headings or H1
H005	info	H1 shared with other pages	headings or H1
H006	warning	H1 too long	headings or H1
H007	info	First heading is not an H1	headings or H1
C001	warning	Canonical missing	canonical
C002	error	Canonical invalid	canonical
C003	info	Canonical points elsewhere	canonical
C004	error	Canonical target not indexable	canonical
C005	error	Canonical chain or loop	canonical
C006	warning	Cross-host canonical	canonical
C007	warning	Canonical scheme mismatch	canonical
C008	info	Canonical target not in crawl	canonical
C009	warning	Canonical differs by slash, case or www	canonical
W001	warning	Thin content	page text
W002	error	Very thin content	page text
W003	warning	Duplicate content	page text
U001	warning	URL too long	URL
U002	warning	Uppercase in URL path	URL
U003	info	Underscores in URL path	URL
U004	warning	Query parameters	URL
U005	warning	Unsafe characters in URL	URL
U006	info	Inconsistent trailing slash	URL
U007	warning	Page served over http	URL
U008	warning	URL variants of the same page	URL
U009	warning	Page too deep	crawl depth
U010	info	Unparseable URL	URL
O001	warning	Og:title missing	Open Graph tags
O002	info	Og:description missing	Open Graph tags
O003	warning	Og:image missing	Open Graph tags
O004	info	Og:url differs from canonical	Open Graph tags
O005	info	Twitter:card missing	Open Graph tags
L001	warning	Lang missing	language
L002	warning	Lang malformed	language
L003	info	Lang differs from site language	language
M001	warning	Images without alt text	images

Pricing

Pay per event, no subscription:

• $1.50 per 1,000 pages ($0.0015 per audited page) (page-audited). You are charged for pages returned in the dataset. Pages excluded by URL patterns and pages beyond your maxPages are free.
• A tiny start fee charged by Apify on every run (about $0.00005).

Job	Pages	Cost
A small site	200	about $0.30
A mid-size site	1,200	about $1.80
A large crawl	50,000	about $75

You can set a maximum cost per run in the run options. The Actor then stops cleanly, tells you how many audited pages were not returned, and charges only for the pages it returned. The site summary and Markdown report still describe every page that was audited. See Apify pricing for plan details.

Limits

• Up to 100,000 pages per run (5,000 inline). Pages whose extracted data is larger than 20,000 characters are skipped and counted in the summary as pagesSkippedOversized.
• The audit itself is fast: auditing 2,000 pages took about a quarter of a second of compute in local runs on a laptop; reading and writing the dataset takes longer than the audit.
• No fetching: the Actor cannot see broken links, redirect targets, page speed, backlinks or rankings. It reports only what your dataset contains.
• Heuristics: lengths, thresholds and the estimated pixel width are rules of thumb. Use profile and rulesDisabled to fit them to your site.

FAQ

Does it crawl or fetch my site? No. It reads a dataset and never makes a request to your website, or to any other service.

Does it call any API or store my data? No. Your pages are processed during the run and written to your own run storage. Nothing is sent anywhere and nothing is kept elsewhere.

Why was a check skipped? The dataset did not contain the data the check needs, for example headings, HTTP status or Open Graph tags. The summary lists the skipped rules and the reason. Turn on Save Markdown in Website Content Crawler to get H1 and word-count checks.

Why are there no Core Web Vitals, backlinks or keyword rankings? Those need live data and third-party sources. This Actor stays within what a crawl dataset can prove.

Can I switch a rule off? Yes, list its code in rulesDisabled. It stops being checked and stops affecting the score.

Why does a noindex page still get issues? Only the title, description and heading findings are reduced to info. Status, canonical, content and URL findings still apply.

Can I audit a CSV? Upload it as a dataset in Apify, or convert it to JSON and paste it into pages.

Related Actors by hiredhandai

• Lead Data Cleaner, Deduper & Scorer: clean, merge and score scraped leads.

Support

Questions, bugs or missing checks: use the Issues tab on this Actor. I answer within 48 hours. If this saved you time, a rating on the Actor page helps others find it.