Site Audit: SEO, Broken Links & AI-Crawler Check
Pricing
from $3.50 / 1,000 page audits
Site Audit: SEO, Broken Links & AI-Crawler Check
Audit a public website over plain HTTP: on-page SEO, broken links, sitemap and robots.txt, llms.txt and AI-crawler rules, security headers and the TLS certificate. One row per page, with each issue's severity, reason and fix.
Pricing
from $3.50 / 1,000 page audits
Rating
0.0
(0)
Developer
GapForge
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
3 days ago
Last modified
Categories
Features
Share
What does Site Audit check?
Site Audit crawls a website you name and reports what is wrong with it, page by page: on-page SEO basics, broken links, sitemap and robots.txt problems, whether AI crawlers may read the site (robots.txt rules for 12 AI bots, plus llms.txt), security headers and the TLS certificate. Each finding comes with a severity, a plain-words reason and a fix. The easiest way to try it: open the Input tab and click Start. The prefilled site returns results in well under a minute.
It works over plain HTTP: no browser, no login, no proxy. It reads the HTML the server sends, the way a search or AI crawler that does not run JavaScript reads it. It identifies itself honestly as GapForgeSiteAudit and follows the site's robots.txt, including Crawl-delay.
Turn on Only new or fixed issues since last run, add a schedule, and it becomes a monitor: a run reports just the pages whose issues changed. That suits agencies and freelancers who watch client sites, developers who want an audit step after each deploy, and site owners who want to hear about a broken link or an expiring certificate before a visitor does.
Why use it
- Several audits in one run. On-page SEO, broken links, sitemap, robots.txt, AI-crawler access,
llms.txt, security headers and TLS, in one dataset. - Findings you can act on. A fixed list of documented issue codes. Each issue states its severity (
error,warningornotice), what was found and how to fix it. - Documented, not a black box. The tables below say exactly what each check tests. The Limits section says what the tool does not do.
- Built for monitoring. Monitor mode remembers the issues it has already reported and tells you what is new and what was fixed.
- Polite by design. It follows robots.txt, keeps to a per-host request rate you choose, and never logs in.
What data do you get?
Two kinds of rows, told apart by type.
One site row per site (free). It is saved before the site's page rows.
| Field | Meaning |
|---|---|
site, url | Scheme and host of the site, and the start address you gave |
reachable | false when the site did not answer at all (also when its TLS certificate was rejected) |
auditedAt | When the audit of this site started (UTC, ISO 8601) |
pagesAudited, pagesDelivered | Page rows produced, and page rows saved and charged |
pagesSkippedByRobots, skippedUrls | Addresses not audited because robots.txt disallows them or the address they redirect to (count, and the first 50) |
pagesFailed, failedUrls | Addresses that did not answer (count, and the first 50). They are not charged |
maxPagesReached, chargeLimitReached | true when the audit stopped at your page limit, or at your maximum charge |
internalLinksNotChecked | Links to pages or files of the site that were not tested, for example because they lie beyond the page limit or the 200-per-site file-link limit. Links that robots.txt disallows are not counted here: disallowed pages are in pagesSkippedByRobots, disallowed files are not counted |
externalLinksChecked, externalLinksNotChecked | Links to other sites that were tested, and links left untested because of the external link limit or because the other host's robots.txt disallows the request, cannot be read, or asks for more than 60 seconds between requests. Both are 0 when Check links to other sites is off |
robotsTxt, crawlDelaySeconds | found, missing, invalid or unreachable, and the Crawl-delay that applies to this tool |
sitemaps, sitemapUrlCount, sitemapInvalidEntries, sitemapTruncated | Sitemap files read, page addresses listed in them, invalid files or entries, and whether the sitemaps were too large to read in full |
llmsTxt, llmsFullTxt, llmsTxtValid | Whether /llms.txt and /llms-full.txt exist, and whether /llms.txt has the basic format |
aiBots, aiBotsBlocked | What robots.txt says to each AI crawler about the home page (allowed, blocked or not-mentioned), and the list of crawlers that may not fetch it |
httpsRedirect | Whether the http:// address redirects to https:// |
securityHeaders, securityHeadersMissing | For six security headers: whether the start page sends each one, and the list of missing ones |
tlsValid, tlsDaysLeft, tlsValidUntil, tlsIssuer | TLS certificate: trusted and matching the host, days until it expires, expiry time, issuing authority |
issues, issueCount, errors, warnings, notices | Site-level issues, and totals for the whole site (site-level issues plus the issues of all audited pages) |
One page row per audited page.
| Field | Meaning |
|---|---|
site, url, finalUrl | The site, the address that was requested and the address after redirects |
status, redirectChain | HTTP status of the final answer, and each redirect hop on the way (url, status) |
responseMs, contentType, bytes | Time to receive the page as seen from the run, its Content-Type and the size of the HTML read |
title, titleLength | Text of the <title> tag and its length in characters |
metaDescription, metaDescriptionLength | Meta description and its length in characters |
canonical, canonicalUrl | self, other or missing, and the address the canonical link points to |
metaRobots, xRobotsTag, noindex, nofollow | The meta robots tag, the X-Robots-Tag header, and what they mean for indexing and link following |
h1Count, lang, hreflangCount | Number of <h1> headings, the <html lang> value, number of hreflang links |
jsonLdTypes, jsonLdErrors | @type values found in JSON-LD blocks, and the number of blocks that are not valid JSON |
imagesMissingAlt | <img> tags with no alt attribute |
internalLinks, externalLinks | Number of different link targets on the same site and on other sites |
brokenLinks | Dead link targets found on the page, as url, status and reason |
inSitemap, depth | Whether the page is listed in the sitemap, and how many clicks it is from the start address |
issues, issueCount, errors, warnings, notices | The page's issues (code, severity, message, fix, changeType) and their counts |
The rows hold facts about pages and servers. Links that start with mailto: or tel: are ignored and never stored.
The checks
What is tested, and where the result shows up.
| Check | What it tests | What it does not test |
|---|---|---|
| Pages and status | Each HTML page reached from the start address by links (and from the sitemap, if switched on): status, redirects, response time | Pages that need JavaScript, a form or a login to appear |
| Titles and descriptions | Presence, length and duplicates of <title> and the meta description among audited pages | Wording quality or keyword use |
| Headings, language, viewport | <h1> count, <html lang>, the viewport meta tag | Heading order below <h1> |
| Canonical and indexing | rel="canonical" (self, other, missing, conflicting), noindex and nofollow in meta robots and X-Robots-Tag | Canonicals sent in an HTTP Link header |
| Structured data | JSON-LD blocks: which @type values are present, and whether each block is valid JSON | Microdata and RDFa; whether the markup qualifies for a rich result |
| Images | <img> tags with no alt attribute | Whether the alt text is good; images added by scripts |
| Mixed content | Images, scripts, stylesheets, frames and media loaded over http:// on an https:// page | Resources requested by scripts or CSS |
| Internal links | Links between pages of the site: each target inside the crawl is fetched; file links (PDF, images and similar) get a light status request | Targets beyond your page limit (counted in internalLinksNotChecked) |
| External links (optional) | Links to other sites: a HEAD request, then GET if HEAD fails. A redirect counts as a working link | Where a redirect ends up; hosts whose robots.txt disallows the request |
| robots.txt | Present, readable and valid; Crawl-delay; rules for this tool | Rules for crawlers not listed below |
| Sitemaps | Sitemaps named in robots.txt plus /sitemap.xml; index files, gzip files and plain-text lists; invalid entries; pages missing from the sitemap | Image, video and news sitemap extensions |
| AI-crawler access (optional) | robots.txt rules for the home page for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Bytespider, Amazonbot and meta-externalagent | Blocks made by a firewall or CDN instead of robots.txt |
| llms.txt (optional) | /llms.txt and /llms-full.txt exist; /llms.txt starts with a # title line and has a Markdown link | The quality or completeness of the file |
| Security headers (optional) | Strict-Transport-Security, Content-Security-Policy, X-Content-Type-Options, X-Frame-Options (or CSP frame-ancestors), Referrer-Policy and Permissions-Policy on the start page; the http:// to https:// redirect | Whether a policy's contents are strict enough; headers on other pages |
| TLS certificate (optional) | Trusted and matching the host; days until expiry; issuer | Protocol versions and cipher strength |
In aiBots, not-mentioned means robots.txt has no group for that crawler, so the general * rules apply to it. aiBotsBlocked lists the crawlers that may not fetch the home page under whichever rules apply to them.
Issue codes
The list is fixed. error means broken for visitors or crawlers, warning means likely to hurt, notice means worth a look.
| Code | Severity | What triggers it | How to fix it |
|---|---|---|---|
HTTP_4XX | error | The start URL or a URL listed in the sitemap answers with a 4xx status (for example 404). | Restore the page, redirect it to its replacement, or remove it from the sitemap and from links. |
HTTP_5XX | error | The start URL or a URL listed in the sitemap answers with a 5xx status (server error). | Check the server logs for this URL and fix the error. |
REDIRECT_LOOP | error | The start URL or a sitemap URL redirects in a loop or through more than 10 redirects. | Point the first URL straight at the final page. |
REDIRECT_OFFSITE | warning | The start URL or a sitemap URL redirects to a different site. | If that is intended, audit the other site instead; otherwise fix the redirect. |
REDIRECT_CHAIN | notice | The page was reached through 2 or more redirects in a row. | Redirect the first URL straight to the final page. |
SITEMAP_URL_REDIRECTS | notice | A URL listed in the sitemap redirects to this page. | List the final URL in the sitemap. |
BROKEN_INTERNAL_LINK | error | The page links to a URL on the same site that answers 4xx or 5xx (401, 403 and 429 are not counted), redirects in a loop, or does not answer. | Fix or remove each link listed in brokenLinks. |
BROKEN_EXTERNAL_LINK | warning | With external link checking on: the page links to a URL on another site that answers 4xx or 5xx (401, 403 and 429 are not counted) or does not answer. | Update or remove each link listed in brokenLinks. |
TITLE_MISSING | error | The page has no <title>, or it is empty. | Add a unique, descriptive <title>. |
TITLE_TOO_LONG | notice | The title is longer than 60 characters. | Shorten the title; search results usually cut long titles. |
TITLE_DUPLICATE | warning | Another audited page that can be indexed has the same title. | Give each page its own title. |
META_DESCRIPTION_MISSING | warning | The page has no meta description, or it is empty. | Add a meta description that summarizes the page. |
META_DESCRIPTION_TOO_LONG | notice | The meta description is longer than 160 characters. | Shorten the meta description; search results usually cut long ones. |
META_DESCRIPTION_DUPLICATE | notice | Another audited page that can be indexed has the same meta description. | Give each page its own meta description. |
H1_MISSING | warning | The page has no <h1> heading. | Add one <h1> that states the page's topic. |
H1_MULTIPLE | notice | The page has more than one <h1> heading. | Keep one main <h1> unless the layout needs more. |
CANONICAL_MISSING | notice | The page has no rel="canonical" link. | Add a canonical link that points to the page's preferred URL. |
CANONICAL_OTHER | notice | The canonical link points to a different URL than the page itself. | Confirm this is intended: search engines are asked to index the other URL instead. |
CANONICAL_MULTIPLE | warning | The page has canonical links that point to different URLs. | Keep one canonical link. |
NOINDEX | notice | The page asks search engines not to index it (meta robots or X-Robots-Tag: noindex). | Remove noindex if the page should appear in search results. |
NOINDEX_IN_SITEMAP | warning | The page is listed in the sitemap but asks not to be indexed. | Remove it from the sitemap, or remove noindex. |
NOFOLLOW | notice | The page tells crawlers not to follow its links (meta robots or X-Robots-Tag: nofollow). This audit does not follow them either. | Remove nofollow if the linked pages should be discovered through this page. |
LANG_MISSING | notice | The <html> tag has no lang attribute. | Add lang, for example <html lang="en">. |
VIEWPORT_MISSING | notice | The page has no <meta name="viewport"> tag. | Add a viewport meta tag so the page scales on phones. |
JSONLD_INVALID | warning | A JSON-LD block (<script type="application/ld+json">) is not valid JSON. | Fix the JSON syntax of the structured data block. |
IMG_ALT_MISSING | notice | One or more <img> tags have no alt attribute (an empty alt="" is accepted). | Add alt text, or alt="" for decorative images. |
MIXED_CONTENT | warning | An https page loads an image, script, stylesheet, frame or media file over http. | Load all resources over https. |
NOT_IN_SITEMAP | notice | The site has a sitemap, and this page can be indexed but is not listed in it. | Add the page to the sitemap, or ignore this if it is left out on purpose. |
SITE_UNREACHABLE | error | The site did not answer (DNS, connection, TLS or timeout error). No page was audited and no page was charged. | Check the address and that the site is online, then run again. |
START_NOT_A_PAGE | warning | The start URL answers, but not with an HTML page (for example a file), so it was not audited as a page. | Start from an address that serves a web page, such as the home page. |
ROBOTS_MISSING | notice | robots.txt answers with a 4xx status (usually 404). | Add a robots.txt, even a short one, and list your sitemap in it. |
ROBOTS_UNREACHABLE | error | robots.txt answers with a server error (5xx) or redirects without end. Crawlers treat the whole site as off limits, and so does this audit. | Make robots.txt answer 200 (or 404 if you have none). |
ROBOTS_INVALID | warning | robots.txt is an HTML page, or has text but no line a crawler understands. | Serve a plain-text robots.txt with User-agent, Allow, Disallow or Sitemap lines. |
ROBOTS_BLOCKS_AUDIT | warning | robots.txt does not allow this tool to fetch the start URL, so no page was audited. | To audit the site, allow the user agent GapForgeSiteAudit in robots.txt. |
CRAWL_DELAY_TOO_LONG | warning | robots.txt asks for more than 60 seconds between requests, so no page was audited. | Lower Crawl-delay for the user agent GapForgeSiteAudit to audit the site. |
SITEMAP_MISSING | warning | No sitemap was found (none listed in robots.txt and none at /sitemap.xml). | Publish a sitemap and list it in robots.txt with a Sitemap line. |
SITEMAP_INVALID_ENTRIES | warning | A sitemap file could not be read, or lists entries that are not valid URLs of this site. | Fix or remove the invalid sitemap files and entries. |
LLMS_TXT_MISSING | notice | With AI-readiness checks on: /llms.txt is absent (or answers with an HTML page). | Optional: publish /llms.txt, a short Markdown guide to the site for AI assistants. |
LLMS_TXT_FORMAT | warning | With AI-readiness checks on: /llms.txt does not start with a "# " title line or has no Markdown link. | Start the file with a line "# Site name" and link the key pages as url. |
AI_BOTS_BLOCKED | notice | With AI-readiness checks on: robots.txt does not let one or more of the listed AI crawlers fetch the home page. | Nothing to fix if that is your choice; otherwise allow those user agents in robots.txt. |
NOT_HTTPS | warning | With security checks on: the start page is served over http, not https. | Serve the site over https. |
HTTPS_REDIRECT_MISSING | warning | With security checks on: the http:// address answers without redirecting to https://. | Redirect all http requests to https with a 301. |
HSTS_MISSING | warning | With security checks on: the https start page sends no Strict-Transport-Security header. | Send Strict-Transport-Security, for example max-age=31536000. |
CSP_MISSING | notice | With security checks on: the start page sends no Content-Security-Policy header. | Add a Content-Security-Policy that lists the sources the site needs. |
CONTENT_TYPE_OPTIONS_MISSING | notice | With security checks on: the start page does not send X-Content-Type-Options: nosniff. | Send X-Content-Type-Options: nosniff. |
FRAME_PROTECTION_MISSING | notice | With security checks on: the start page sends neither X-Frame-Options nor a Content-Security-Policy with frame-ancestors. | Send X-Frame-Options: SAMEORIGIN or a CSP frame-ancestors rule. |
REFERRER_POLICY_MISSING | notice | With security checks on: the start page sends no Referrer-Policy header. | Send Referrer-Policy, for example strict-origin-when-cross-origin. |
PERMISSIONS_POLICY_MISSING | notice | With security checks on: the start page sends no Permissions-Policy header. | Send a Permissions-Policy that switches off the browser features the site does not use. |
TLS_INVALID | error | With security checks on: the TLS certificate is expired, not trusted or does not match the host name. No page can then be read, so the site row also says SITE_UNREACHABLE and no page is audited. | Renew or reinstall the certificate. |
TLS_EXPIRES_SOON | warning | With security checks on: the TLS certificate expires in 30 days or fewer. | Renew the certificate. |
How to use it (step by step)
- Open the Input tab. The prefilled site already works.
- In Sites to audit, add one address per site, for example
https://www.your-site.com. The audit starts there and stays on that host (with or withoutwww). - Set Maximum pages per site. You pay per page row, so this also caps the cost.
- Optional: switch on Check links to other sites, and switch Security checks or AI-readiness checks off if you do not need them.
- For monitoring, turn on Only new or fixed issues since last run and add a schedule (weekly works well). Use a different State key for each schedule so their memories stay separate.
- Click Start. Results appear in the Output tab as tables (Overview, Pages, Issues, plus All fields) and can be exported as JSON, CSV or Excel.
Monitor mode
With Only new or fixed issues since last run on, the tool remembers which issues each page had. A page row is saved (and charged) when the page has an issue it did not have last time, or when an issue it had is gone. Each issue then carries a changeType: new, unchanged or resolved. A broken-link issue is remembered together with its list of dead links: when that list changes (a link breaks, or one of several is fixed), the issue is reported as new again with the current list, and resolved appears when the page has no such broken link left. On a page, resolved can also mean that the dead link was not tested in this run (it lay beyond the page, file-link or external-link limit, or Check links to other sites was switched off); when a limit was the cause, the site row's internalLinksNotChecked or externalLinksNotChecked counts those links. AI_BOTS_BLOCKED works the same way with its list of crawlers. A page with no change is checked but not saved and not charged. The first run with a state key reports each current issue as new. In a normal run changeType is snapshot.
The site row is saved in each run, free, with its counts for the whole site. A site-level issue is reported as resolved only when its check ran in this run and no longer finds it; when the site cannot be reached, or a check is switched off, the earlier site-level issues are kept as they were. The memory is a named key-value store in your own Apify account called site-audit-state-KEY, where KEY is your state key. Delete it to start fresh.
How much does a site audit cost?
You pay per page audited, after the page row is saved:
- $0.005 per page row, which is $5 per 1,000 pages audited (less on paid Apify plans, down to $3.50 per 1,000 pages; the Pricing tab shows your price).
- Apify also charges a small start fee of $0.00005 per run.
- The site summary row is free. A site that cannot be reached and a page that does not answer add no page charge; such a run costs only the start fee.
- A start address or sitemap address that answers with an error, loops or leaves the site is a page row too (it carries the finding) and is charged like one.
Examples: auditing a 100-page site costs about $0.50. A weekly monitor run over the same site that finds 4 pages with new or fixed issues costs about $0.02. Apify's free plan includes $5 of monthly credit, enough to audit 999 pages in one run. The tool stops cleanly when it reaches the maximum charge you set for the run and tells you so in the status message.
Input example
{"startUrls": ["https://www.your-site.com"],"maxPagesPerSite": 100,"useSitemap": true,"maxRequestsPerSecond": 2,"checkExternalLinks": false,"maxExternalLinksPerSite": 200,"checkSecurityHeaders": true,"checkAiReadiness": true,"onlyChanges": false,"stateKey": "weekly-my-site"}
Output example
A page row:
{"type": "page","site": "https://example.com","url": "https://example.com/pricing","finalUrl": "https://example.com/pricing","status": 200,"redirectChain": [],"responseMs": 182,"contentType": "text/html; charset=utf-8","bytes": 18734,"title": "Pricing","titleLength": 7,"metaDescription": null,"metaDescriptionLength": 0,"canonical": "self","canonicalUrl": "https://example.com/pricing","metaRobots": null,"xRobotsTag": null,"noindex": false,"nofollow": false,"h1Count": 1,"lang": "en","hreflangCount": 0,"jsonLdTypes": ["Product"],"jsonLdErrors": 0,"imagesMissingAlt": 2,"internalLinks": 14,"externalLinks": 3,"brokenLinks": [{"url": "https://example.com/old-plan", "status": 404, "reason": "status 404"}],"inSitemap": true,"depth": 1,"issues": [{"code": "BROKEN_INTERNAL_LINK", "severity": "error", "message": "1 broken link(s) to this site, first: https://example.com/old-plan (status 404).", "fix": "Fix or remove each link listed in brokenLinks.", "changeType": "snapshot"},{"code": "META_DESCRIPTION_MISSING", "severity": "warning", "message": "The page has no meta description, or it is empty.", "fix": "Add a meta description that summarizes the page.", "changeType": "snapshot"},{"code": "IMG_ALT_MISSING", "severity": "notice", "message": "2 image(s) have no alt attribute.", "fix": "Add alt text, or alt=\"\" for decorative images.", "changeType": "snapshot"}],"issueCount": 3,"errors": 1,"warnings": 1,"notices": 1}
The values above illustrate the format.
Integrations and API
Run it from the Apify API, the Apify CLI or the Apify clients for Python and JavaScript, on a schedule, or from automation tools that connect to Apify (for example Make, n8n or Zapier). A webhook can notify your system when a run finishes, so new issues can flow straight into a ticket tracker, a chat channel or a spreadsheet. AI agents can call it through the Apify MCP server; the input and output schemas tell them what to send and what comes back.
FAQ and troubleshooting
Is it legal?
You audit a site you name. The tool fetches public pages the way a search crawler does, follows the site's robots.txt, keeps to a polite request rate and never logs in. Audit sites you own or are allowed to test, and check your own use against the laws and site terms that apply to you.
Will it overload my site?
No more than a small crawler would. Requests to one host are sent one at a time, at the rate set in Maximum requests per second (2 by default, 5 at most). If your robots.txt sets a Crawl-delay that is slower, the tool waits that long between requests.
Why are some pages missing?
Four possible reasons. The first three are shown in the site row. robots.txt disallows them (skippedUrls). The audit reached your page limit (maxPagesReached). The page did not answer (failedUrls). Or reaching the page takes JavaScript, a form or a login, which this tool does not use. Pages listed in the sitemap are found even when nothing links to them, as long as Start from the sitemap too is on.
How do I let the tool audit a site that blocks crawlers?
Add a group for it to the site's robots.txt: User-agent: GapForgeSiteAudit, then Allow: /. If a firewall or CDN blocks unknown crawlers, allow that user agent there too.
Why is a link reported as broken when it opens in my browser?
The target answered the tool with an error status, or did not answer within the time limit. A link to a host that does not answer at all, not even for its robots.txt, is listed as broken too. Some sites refuse automated requests; answers 401, 403 and 429 are therefore not counted as broken. The reason field in brokenLinks says what happened.
The run finished with 0 page rows in monitor mode.
No page had an issue appear or disappear since the last run with the same state key. No page was charged; the run costs only the start fee. Changes to site-level issues (for example a certificate entering its last 30 days) are still in the free site row, tagged new or resolved, and the status message then says "the site row of 1 site(s) has new or resolved issues".
It stopped early.
The status message says why. Most often the run reached your maximum charge; raise the limit and run again.
Is this an official product of a search engine or an AI company?
No. This tool is not affiliated with or endorsed by Apify, Google, OpenAI, Anthropic, Perplexity, Apple, Amazon, Meta, Common Crawl, ByteDance or any site it audits. The crawler names in the AI-crawler check are the user agents those companies publish.
Something looks wrong.
Open an Issue on this page with the run ID. We reply within one business day and fix confirmed bugs in the next build.
How URLs are compared
- The fragment (
#section) is dropped; scheme and host are lower-cased; default ports are dropped. /pageand/page/are treated as two different addresses, as a server treats them. If one redirects to the other, they become one page row.- The query string is kept as written, so
?page=2is a different address from?page=3. - A host with and without
www.is the same site. Other subdomains are separate sites: add them as their own start address.
Limits
- No JavaScript rendering: the tool reads the HTML the server sends. Content added by scripts is not seen.
- No page speed measurement, no accessibility scoring and no judgement of rich-result eligibility.
responseMsis the time one request took from the run, not a speed score. - At most 20 sites per run, 2,000 pages per site and 500 external links per site.
- robots.txt is followed, so disallowed pages are skipped and listed as skipped. A site that asks for more than 60 seconds between requests is not audited.
- One page is read up to 5 MB, and one sitemap file up to 20 MB after decompression. At most 50 sitemap files and 50,000 sitemap addresses are read per site; beyond that
sitemapTruncatedistrue. - File links (PDF, images and similar) are status-checked, at most 200 per site, and are never audited as pages.
- Duplicate titles and descriptions are compared among the pages audited in this run.
- Requests to one host are sent one at a time, so large sites take longer by design.
- A site's rows are saved when the audit of that site is finished. In our tests a 2,000-page audit took 25 to 33 minutes with Maximum requests per second set to 5; the default of 2 was not timed at this size. Sites are audited one after another, so keep the run timeout long enough for all sites in the run: if a run is aborted or times out in the middle of a site, the rows of that site and of the sites after it are not saved and not charged.
- Sitemap files are read before the first page, one at a time, even when Start from the sitemap too is off (they are needed for the sitemap checks). A site with slow or very large sitemap files can add several minutes to a run.
brokenLinkslists at most 100 links per page. Titles and meta descriptions are stored up to 1,000 characters;titleLengthandmetaDescriptionLengthgive the full length.jsonLdTypeslists at most 50 types.