SEO Audit Tool — Sitemap Crawler & Lighthouse Site Checker

Give it one website address. Get a full technical SEO audit that tells you what is wrong in plain English — not 150 raw Lighthouse audits you have to interpret yourself.

This Actor is a site crawler and an audit engine in one. It parses robots.txt, walks every sitemap the site declares, and runs Google Lighthouse — the engine behind Chrome DevTools and PageSpeed Insights — on the pages it finds. Then it does the part other Lighthouse wrappers leave to you: it writes the verdict.

LCP 4.5s — fails Google's mobile threshold of 4.0s; TBT 871ms — fails Google's mobile threshold of 600ms; 1 render-blocking resource

Every audited page gets a headline like that, a critical / needs-work / good verdict, and a ranked fix-first list. Every site gets one summary row naming the five worst pages and the five things to fix, so you can read one row and know where you stand.

No Chrome extension, no per-page copy-pasting, no API key.

Built for SEO audits, site migrations, client reporting, Core Web Vitals tracking, and pre-launch QA. Runs on the Apify platform, so you can schedule it, call it from the API, or wire it into your own pipeline.

Unofficial. This Actor runs the open-source Lighthouse project (Apache-2.0) and is not affiliated with, endorsed by, or sponsored by Google. It audits publicly reachable pages only, respects robots.txt, and identifies itself honestly. You are responsible for complying with the target site's Terms of Service and with the laws that apply to you.

---

Why use this Actor?

• It tells you the answer, not just the data — every page carries a one-line verdict written against Google's own published Core Web Vitals thresholds, plus a ranked list of what to fix first. You do not need to know what "Total Blocking Time" means to use the output.
• One site verdict row — domain-summary names how many pages are critical, the five worst pages with their headlines, and the five highest-impact site-wide fixes with the milliseconds each one recovers.
• Audits the crawl layer Lighthouse cannot see — a robots-txt row per site reporting the Disallow rules, Crawl-delay, and declared sitemaps, plus an optional row for every URL in the sitemap with lastmod, changefreq, priority, duplicate detection, off-host detection, and whether robots.txt actually allows it. Lighthouse reports on none of this.
• One input, hundreds of rows — a single domain typically yields ~50 rows per page. An 8-page site returns roughly 400 rows in one run.
• Finds the pages for you — reads sitemap.xml first (including sitemap indexes), falls back to internal-link discovery. You do not supply a URL list.
• Four audit categories in one pass — performance, SEO, accessibility, best practices. Turn off what you do not need.
• Site-wide summaries — average score per category and the most frequent issues across the whole site, as their own rows. No spreadsheet work on your side.
• Compare several sites in one run — put your client and their competitors in together and get a ranked league table per category. See Compare a client against competitors.
• Mobile and desktop side by side — audit one device or both. Both means two full sets of scores, tagged by formFactor, so you can see exactly where mobile falls behind.
• Never charged for failures — pages that time out, refuse the connection, or are blocked by robots.txt cost you nothing.
• Structured output — one flat row per finding, ready for CSV, Excel, Google Sheets, or a database.

---

How to use it

1. Put one or more website addresses in Websites to audit (https://apify.com).
2. Set Pages to audit per website — how deep to go. Start at 3–10, raise it once you see the shape of the output.
3. (Optional) Trim Audit categories to just the ones you care about — fewer categories, faster runs, fewer rows.
4. Choose Devices: mobile (Google's default for ranking), desktop, or both. Both doubles the audit time and the number of rows.
5. Start the run and export from the Output tab as JSON, CSV, Excel, or HTML.

---

Compare a client against competitors

This is what the Actor is best at. Put the site you work on and the sites you are measured against into startUrls in the same run. Every site is audited identically, and the run adds comparison rows: one per category per site, with the rank, the leader, and how many points behind you are.

{
    "startUrls": [
        "https://your-client.com",
        "https://competitor-one.com",
        "https://competitor-two.com",
        "https://competitor-three.com"
    ],
    "maxUrlsPerDomain": 5,
    "categories": ["performance", "seo"],
    "formFactors": ["mobile"],
    "maxRunSecs": 1800,
    "maxResults": 800,
    "disclaimerAck": true
}

A comparison row looks like this:

{
    "rowType": "comparison",
    "domain": "your-client.com",
    "category": "performance",
    "formFactor": "mobile",
    "auditTitle": "performance ranking on mobile: #3 of 4",
    "score": 61,
    "displayValue": "#3 of 4 — 61/100, 27 point(s) behind competitor-one.com (88)",
    "rank": 3,
    "comparedDomains": 4,
    "bestDomain": "competitor-one.com",
    "bestScore": 88,
    "gapToBest": 27
}

Filter the dataset to rowType = comparison and you have the slide: who leads each category, and by how much. Up to 10 sites per run. Comparison rows appear only when at least two sites returned results — a single-site run produces none.

Want the mobile-versus-desktop gap too? Set "formFactors": ["mobile", "desktop"]. Every score row, summary row and comparison row is tagged with formFactor, and rankings are produced per device, so mobile and desktop are never averaged together.

---

Input

Field	Type	Description	Default
startUrls	array	Websites to audit. A domain or a specific page. Two or more sites also produce a ranked comparison (max 10)	https://example.com
maxUrlsPerDomain	integer	Pages to audit per website (max 200)	3
categories	array	performance, seo, accessibility, best-practices	all four
formFactors	array	Devices to audit on: mobile, desktop, or both	["mobile"]
includePassedAudits	boolean	Also return audits the page already passes	false
pageTimeoutSecs	integer	Per-page timeout, seconds (max 180)	60
maxRunSecs	integer	Total run time budget, seconds	900
maxResults	integer	Hard cap on result rows (max 5000)	500

Example input:

{
    "startUrls": ["https://example.com"],
    "maxUrlsPerDomain": 10,
    "categories": ["performance", "seo"],
    "formFactors": ["mobile"]
}

The earlier single-device field "formFactor": "mobile" still works, so saved tasks and API calls written against the first release keep running unchanged.

Invalid input ends the run as SUCCEEDED with a notice row.

---

Output

Every row is one finding. Eight row types come back in the same dataset, tagged by rowType.

The site verdict — one row per site per device. If you read nothing else, read this one:

{
    "rowType": "domain-summary",
    "domain": "apify.com",
    "headline": "apify.com on mobile: 3 page(s) critical, 0 need work, 0 healthy across 3 audited page(s). Scores — performance 54 · accessibility 99 · best-practices 57 · seo 85.",
    "verdict": "critical",
    "score": 53,
    "worstPages": [
        "https://apify.com/ (45/100) — LCP 36.2s — fails Google's mobile threshold of 4.0s; Page is affected by an invalid robots.txt; Page has uninformative link text",
        "https://apify.com/actors (57/100) — LCP 4.5s — fails Google's mobile threshold of 4.0s; TBT 871ms — fails Google's mobile threshold of 600ms; 1 render-blocking resource"
    ],
    "immediateActions": [
        "1. Reduce unused JavaScript — affects 3 of 3 page(s), ~8100ms recoverable",
        "2. Remove duplicate modules in JavaScript bundles — affects 3 of 3 page(s), ~600ms recoverable"
    ],
    "pagesCritical": 3,
    "formFactor": "mobile"
}

The crawl configuration — one robots-txt row per site, covering what Lighthouse never looks at:

{
    "rowType": "robots-txt",
    "domain": "apify.com",
    "headline": "robots.txt: 0 Disallow rule(s), 1 sitemap(s) declared",
    "verdict": "good",
    "robotsFound": true,
    "disallowRules": [],
    "crawlDelaySecs": null,
    "declaredSitemaps": ["https://apify.com/sitemap.xml"],
    "sitemapUrlsFound": 50,
    "duplicateSitemapUrls": 0,
    "offHostSitemapUrls": 0
}

A sitemap URL record — opt-in, one per URL the sitemaps declare:

{
    "rowType": "sitemap-url",
    "domain": "apify.com",
    "url": "https://apify.com/about",
    "headline": "Sitemap URL has no lastmod",
    "verdict": "needs-work",
    "sitemapUrl": "https://apify.com/sitemap/pages.xml",
    "lastmod": null,
    "lastmodAgeDays": null,
    "changefreq": null,
    "priority": null,
    "robotsAllowed": true,
    "duplicateCount": 0,
    "wasAudited": true
}

A page score — one per page per category, now carrying the page verdict:

{
    "rowType": "page-score",
    "domain": "apify.com",
    "url": "https://apify.com/",
    "category": "performance",
    "auditTitle": "Performance",
    "score": 45,
    "headline": "LCP 36.2s — fails Google's mobile threshold of 4.0s; Page is affected by an invalid robots.txt; Page has uninformative link text",
    "verdict": "critical",
    "topActions": [
        "Fix: page is affected by an invalid robots.txt",
        "Fix: page has uninformative link text",
        "Reduce unused JavaScript — saves ~8100ms"
    ],
    "lcpMs": 36222.39,
    "cls": 0,
    "tbtMs": 436,
    "renderBlockingCount": null,
    "formFactor": "mobile",
    "lighthouseVersion": "12.8.2",
    "collectedAt": "2026-08-26T02:46:45.843Z"
}

An individual finding — what to actually fix, with the size of the win:

{
    "rowType": "audit",
    "domain": "example.com",
    "url": "https://example.com/",
    "category": "performance",
    "auditId": "render-blocking-resources",
    "auditTitle": "Eliminate render-blocking resources",
    "auditDescription": "Resources are blocking the first paint of your page.",
    "score": 0.42,
    "displayValue": "Potential savings of 320 ms",
    "savingsMs": 320,
    "savingsBytes": null
}

Site-wide rows — domain-category-score (average per category across the site, per device) and domain-top-issue (the findings that recur on the most pages, per device).

Comparison rows — comparison, produced only when two or more sites returned results. One per category per device per site, carrying rank, bestDomain and gapToBest. See Compare a client against competitors.

You can download the dataset as JSON, CSV, Excel, or HTML.

---

Data fields

Field	Format	Description
rowType	text	domain-summary / robots-txt / sitemap-url / page-score / audit / domain-category-score / domain-top-issue / comparison
domain	text	Website the row belongs to
url	link	Page audited (empty on site-wide rows)
headline	text	The verdict in one sentence — what is wrong with this page, site or URL
verdict	text	critical, needs-work or good
topActions	array	Page rows: the ranked fixes for that page, with the milliseconds each recovers
immediateActions	array	domain-summary: the five highest-impact site-wide fixes
worstPages	array	domain-summary: the five lowest-scoring pages with their headlines
pagesCritical / pagesNeedWork / pagesHealthy	number	domain-summary: page counts by verdict
lcpMs / cls / tbtMs	number	Core Web Vitals measured for the page
renderBlockingCount	number	Render-blocking resources found on the page
robotsFound	boolean	robots-txt: whether the site serves a robots.txt at all
disallowRules / allowRules	array	robots-txt: the rules that apply to crawlers
crawlDelaySecs	number	robots-txt: declared Crawl-delay, honoured by this Actor
declaredSitemaps	array	robots-txt: sitemaps the site points crawlers at
sitemapUrlsFound / duplicateSitemapUrls / offHostSitemapUrls	number	robots-txt: sitemap health counters
sitemapUrl	link	sitemap-url: which sitemap file listed this URL
lastmod / lastmodAgeDays / changefreq / priority	text / number	sitemap-url: the metadata the sitemap declares
robotsAllowed	boolean	sitemap-url: whether the site's own robots.txt permits this URL
sameHost	boolean	sitemap-url: whether the URL points at the same host
duplicateCount	number	sitemap-url: extra times this URL was listed
wasAudited	boolean	sitemap-url: whether this run ran Lighthouse on it
category	text	performance, seo, accessibility, best-practices
auditId	text	Lighthouse audit identifier, e.g. render-blocking-resources
auditTitle	text	Human-readable name of the score or finding
auditDescription	text	What it means and why it matters
score	number	0–100 for page scores, 0–1 for individual audits
displayValue	text	Lighthouse's own summary, e.g. "Potential savings of 320 ms"
savingsMs	number	Estimated time saved by fixing it
savingsBytes	number	Estimated bytes saved by fixing it
formFactor	text	mobile or desktop — the device this row was measured on
rank	number	Comparison rows: this site's position for that category and device
comparedDomains	number	Comparison rows: how many sites were ranked
bestDomain	text	Comparison rows: the site that leads that category
bestScore	number	Comparison rows: the leader's score
gapToBest	number	Comparison rows: points behind the leader (0 for the leader)
affectedUrls	number	Summary rows: pages the score or issue covers
lighthouseVersion	text	Engine version used for the run
collectedAt	date	Timestamp

---

Pricing / Cost estimation

$0.03 per result row. No subscription, no monthly fee.

What is charged: every delivered result row — domain-summary, robots-txt, sitemap-url, page-score, audit, domain-category-score, domain-top-issue and comparison. The summary, crawl-configuration and comparison rows are charged because they are results in their own right: they are the analysis you would otherwise do by hand, delivered ready to paste into a report.

Sitemap inventory is off by default. A large sitemap can hold thousands of URLs, and every delivered row is billable, so you switch it on deliberately with Include full sitemap inventory and cap it with Max sitemap URLs per website (default 100). Leave it off and you are never charged for it.

What is never charged: pages that time out or refuse the connection, URLs blocked by robots.txt, sites that produce no results, and every status or notice row. If a run stops early on its limit, you are only charged for what you actually received.

Measured on real runs:

What you audit	Rows returned	Cost
1 page, four categories, mobile	~50–70	~$1.50–2.10
The same page on mobile and desktop	~2× the rows	~2× the cost
Small site (8 pages), mobile	~400	~$12.00
Trimmed to 2 categories	roughly half the rows	roughly half the cost
4 sites compared, 2 categories, mobile	comparison rows add 8 (2 categories × 4 sites)	~$0.24 on top of the audits

Two ways to control spend: lower Pages to audit per website, or set Max result rows — the run stops cleanly at that number and tells you it stopped. You can also set a maximum cost per run in the Actor's Run options; the run then finishes successfully with everything collected up to that point rather than failing.

Free-plan users get a sample of up to 25 rows per run (auditing up to 3 pages per site), then the run ends successfully. The sample is not simply the first 25 rows produced: the run finishes auditing every site first, then delivers the highest-value rows — the site verdict row first, then site-vs-site comparison rows, site summary scores, top recurring issues, the robots.txt analysis, page scores, a taste of the individual audit findings, and finally sitemap URL records. A closing message states exactly what the full dataset contained (for example, how many individual findings were left out). Upgrade your Apify plan to receive everything.

---

Tips

• Fewer categories, more pages. If you only care about search, run ["seo"] and raise maxUrlsPerDomain — you get broader coverage for the same spend.
• Mobile is the number that matters for Google ranking. Audit desktop separately when you are chasing a specific complaint.
• Scores move between runs. Lighthouse measures a live page over a real network, so performance scores vary a few points run to run. Compare trends, not single numbers.
• One device for routine monitoring, both when something breaks. Weekly tracking on mobile keeps the cost and run time down; switch both on when a client reports a problem you cannot reproduce.

---

Set up a weekly automatic audit

A single audit tells you where a site stands today. A scheduled audit tells you when it slips — a deploy that adds 400 KB of JavaScript, an image nobody compressed, an alt attribute dropped in a redesign. You find out on Monday morning instead of when the rankings move.

Scheduling is built into Apify and needs nothing from you but a few clicks:

1. Run the Actor once with the input you want to repeat, so you know the run time and the row count.
2. On the run page, click Create task (or open the Actor → Tasks → Create new task). A task stores that exact input under a name like client-weekly-seo.
3. Open Schedules in the left sidebar → Create new schedule.
4. Add your task to it, set the cadence — Weekly, or a cron expression such as 0 6 * * 1 for every Monday at 06:00 — pick your timezone, and enable it.

Every run writes its own dataset, so the history builds up on its own. A few ways teams use it:

• Regression alarm — the same 5–10 pages every Monday. Filter each dataset to rowType = domain-category-score and chart the score over time.
• Competitor tracking — the multi-site input from above, run monthly. The comparison rows become a standing scoreboard of you against the field.
• Client reporting — one schedule per client, each with its own task, exported to Google Sheets through Apify's integrations or the API.

Budget it before you enable it: a schedule bills every time it fires. Multiply the row count of your test run by $0.03, then by how often it runs. Set Max result rows on the task as a hard ceiling so a site that suddenly grows a thousand sitemap entries cannot surprise you.

---

Limitations

• Audits publicly reachable pages only — no login, no cookie injection, no bypassing of protections.
• Pages behind heavy bot protection may refuse the audit. Those rows come back marked as failed and are not charged.
• Page discovery uses sitemap.xml when present and internal links otherwise; pages reachable only through search boxes or JavaScript-only navigation may be missed.
• robots.txt is respected. Disallowed URLs are skipped and not charged.

---

FAQ and support

Is this the same as PageSpeed Insights? It runs the same Lighthouse engine, but across your whole site in one go and returns the results as data instead of one report per page.

Why do my scores differ from PageSpeed Insights? PSI blends lab data with real-user field data from the Chrome UX Report. This Actor returns lab data only, measured at run time.

Something is wrong or missing. Open an issue on the Actor's Issues tab — it is read and answered.

⭐ If this audit saved you time, a sentence about what you used it for helps the next person decide whether it fits their case.

Works well with

• subfinder-osint — Find every subdomain of the site you audited, so no forgotten host stays unmeasured.
• whois-domain-lookup — Check registrar, domain age and expiry for the sites you just scored.

Every run of this Actor also returns these suggestions in the nextSteps field of its summary row.