SEO audit for a whole website: broken links, redirects, duplicate titles and descriptions, sitemap and indexing problems. Get a health score and a fix for every issue, and see what changed since the last run. Up to 5,000 pages per run.
A readable Report is the first view of every run: health score, errors, warnings and
notices, what changed since the last run, and every issue with where it is and how to
fix it.
An Issues view with one row per problem, errors first, each with the exact value
that is wrong and how to fix it.
A Pages view with one row per page, and a Changes view with one row per change
since the last run on the same site.
28 checks, including broken links, redirect loops, duplicate titles and content,
hreflang return links, orphan pages and sitemap URLs blocked by robots.txt.
One price per audit, set by how many pages were checked. Runs under 10 pages are free,
and a run that cannot reach your site is free.
Corrected before release
A pre-publish audit went through the whole product. Nothing below ever reached a buyer; the
list is here so the record of what changed is complete.
What you are charged.
A run's maximum cost now limits how many pages it checks: if the cap is below the price
for Max pages, the run checks up to 1,000 or 250 pages (or 9, free) and says so. An audit
fee the cap does not cover is never sent or reported as charged.
Paid runs no longer warn that billing stopped.
The cost estimate no longer quotes a price for pages 1-9.
A restarted run that was already charged is never described as free or uncharged.
What the run tells you.
Every successful run ends with a one-line status: pages, health score, changes, early stop
and charge.
A run stopped by its time limit before any page says so, instead of blaming the server.
Stop warnings no longer give a page count that differs from the final one.
Memory advice no longer asks a 2 GB run for more memory.
The advice for a site that refuses the crawl no longer asks to allow the Actor.
A run with invalid input still writes its report and summary.
What gets crawled.
A crawl that starts at a hosted address, such as name.wordpress.com, stays on that one
address instead of spreading over the hosting platform.
A restarted run checks its full page limit again; Crawlee's saved request count no longer
cuts it short.
Resurrecting a run that already finished no longer writes every row twice; the run stops
and says where its results are.
A short run timeout no longer stops the crawl before the first page; the time kept for
later steps shrinks with the timeout, and under 4 minutes links to other sites are not
checked.
Links to other sites keep to 2 requests a second per host, redirects included.
What gets reported.
Duplicate title, content and meta description are no longer reported on pages set to
noindex or canonicalised away, or on their originals.
"Not indexable" reads the robots directives exactly, so max-image-preview:none is no
longer read as noindex.
"Page missing from sitemap" is only reported on hosts whose sitemap was read.
The orphan page check stays on when the only skipped links were add-to-cart and similar
addresses.
The Report says "some pages found were left out" when your patterns, the link depth limit,
the crawl-trap filter or robots.txt kept a page out, instead of claiming every page found
was checked. The note about skipped pages names the crawl-trap filter, and now appears on
sites with no sitemap too.
The Report says when robots.txt disallowed pages, and when it could not be read at all.
The Report's change table shows the changes that matter first, so a page set to noindex is
not pushed out by URLs that moved, and every kind of change gets at least one row.
A very long address is shown as text, cut in the middle so the end still tells two of them
apart, and one page cannot make the Report too large to save.
The share bar is left empty for sitemap URLs blocked by robots.txt, which are not pages
that were checked.
All four issue-table headings are set in the same font.
Change history.
Runs with different URL patterns, link depth or sitemap setting on the same site each
compare against their own last run.
A page no longer found is reported once, even when the crawl boundary moved.
A site typed with and without www. no longer reports every page as removed and re-added:
the two spellings share one history store and each compares against its own last crawl.
Change rows are written before the snapshot is saved.
The run log counts "today" and "yesterday" on UTC dates.
Outputs and the form.
summary.json lists the run's dataset ids (datasets) and, when it applies, the
maximum-cost limit (maxCostCap).
The Pages output link no longer promises fields its view leaves out, and the Issues and
Changes links say they need an API token outside the Console.
The input form opens with what a run does and what it costs, and the crawl settings are in
a section of their own.
The Report's monospace font is IBM's own subset of IBM Plex Mono, shipped with IBM's
licence text. The report looks the same.
Corrected in review round 2, before release
Seven readers went over the pre-publish audit's own changes, and independent skeptics tried
to refute every serious finding. Nothing below reached a buyer either.
What you are charged.
Pages your site refuses are not charged for. A page that answers 401, 403 or 429 was not
read, so a run behind bot protection is free unless it still read 10 pages or more.
A free run gives one page count that matches the other: "Only 8 of the 12 pages crawled
could be read, so this run is free."
A price band the run's own pricing does not list is no longer reported as charged.
A run whose charge receipt was saved but not listed is still charged, and a run killed
while its report was being written charges normally when it is started again.
What gets reported.
Duplicate title, meta description and content are listed as not checked, not as passed,
when there were not two indexable pages to compare — and a later run no longer says they
were "fixed".
The orphan page check is skipped when your robots.txt kept pages out of the crawl, because
a page this crawler may not read can be one search engines reach.
A noindex written as noindex;nofollow, or after a directive the Actor does not
recognise, is now read as not indexable.
A run that could not reach its start URL names what actually happened: a 404 homepage is no
longer reported as a redirect loop.
The Report's first note about a maximum-cost limit says the same three things the run log
does: how many pages it will check, that the run is free when it is, and what to raise the
maximum cost to.
The change table keeps a row for every kind of change, so a site-wide title rewrite no
longer hides the pages that appeared or disappeared.
Change history and scope.
A start address written with a trailing dot (https://example.com./) is crawled instead of
putting the whole site out of scope.
An unreadable snapshot for one set of settings falls back to the site's last crawl, so that
run's changes are still reported.
A history store that refuses to delete an old record no longer reports the snapshot it just
saved as lost.
Runs, time and memory.
A run's per-request timeouts come out of the time it actually has, so a slow page is less
likely to end a short run before its results are saved — and under about 40 seconds the run
log says the risk is there.
A run started by an older client reads its real memory limit, so the memory guard stops it
at the right point.
Resurrecting a finished run says what that audit found before saying it already finished.
The run's one-line status says when some pages found were left out.
The form and the documentation.
The Website URL help text says an IP address is not accepted, and the FAQ says which
addresses are refused before the crawl starts.
The FAQ names every browser whose headers the crawl can send, not only two of them.
The Changes view is described as it is ordered: the URLs no longer found first, then
everything else in URL order.
