Whole-Site PageSpeed & Core Web Vitals Audit (via Sitemap)

Who it's for: SEO consultants, agencies and site owners who want a site-wide speed report, not page-by-page checks.

The problem it solves: A single PageSpeed test doesn't tell you how the whole site performs, and collecting URLs first is tedious. Give this Actor one homepage: it finds the pages, samples them across the site, audits each and summarizes what to fix first.

Use cases:

• Site-wide Core Web Vitals health check for SEO
• Prospect audits and agency client reports
• Find the worst pages and the fixes that affect the most pages
• Track site performance over time on a schedule

Quick example: {"siteUrl":"https://crawlee.dev","maxPages":10} → one row per page (scores, Core Web Vitals, fixes) plus a SITE_SUMMARY with averages, CWV pass rate, worst pages and top recurring opportunities.

Price: $3.00 per 1,000 page audits + $0.02 per site summary. A 10-page audit costs about $0.05. Comparable PageSpeed/Lighthouse Actors in the Store charge $2 to $80 per 1,000 URLs.

Enter one website and get a Google PageSpeed Insights / Lighthouse audit of its pages, plus a site-wide summary.

It finds the site's pages automatically from robots.txt and its sitemaps (including sitemap indexes and gzipped sitemaps), samples pages across the whole site (homepage first), runs Google's official PageSpeed Insights API v5 on each page, and then tells you:

• Average scores for Performance, Accessibility, Best Practices and SEO
• Median lab metrics: LCP, TBT, CLS, FCP
• Core Web Vitals pass rate from real-user (CrUX) data, plus the origin-level verdict
• Worst pages, so you know where to start
• Top recurring opportunities: fixes that affect the most pages, with total estimated savings
• Recurring failed audits: accessibility, SEO and best-practice problems that repeat across the site

Each page also gets its own full result row: scores, lab and field Core Web Vitals, top opportunities, and failed audits.

How it works

1. Discover: robots.txt → sitemaps → sitemap indexes → page URLs. Images, PDFs and other domains are skipped. Your include/exclude regex is applied.
2. Sample: the homepage plus pages spread evenly across the sitemap, so 10 pages cover different sections rather than the first 10 blog posts.
3. Audit: PageSpeed Insights for each page on mobile, desktop or both, with retries on rate limits and server errors.
4. Summarize: the site summary is saved as SITE_SUMMARY in the key-value store.

There's no browser to run and no proxies. Google runs Lighthouse on its own servers, so the numbers match pagespeed.web.dev.

Input

Field	Default	Description
siteUrl	https://crawlee.dev	Homepage or a sitemap URL
maxPages	10	Pages to audit, sampled across the sitemap
includeHomepage	true	Audit the homepage first
includeRegex / excludeRegex	–	e.g. only /blog/, skip /tag/
strategy	mobile	mobile, desktop or both (2 audits per page)
categories	all four	Lighthouse categories
apiKey	–	Optional: your own free Google API key, for more than 30 audits per run
maxConcurrency	4	Parallel audits

API key

You don't need a key to start. Runs without one use the Actor's built-in key, limited to 30 audits per run. Extra pages are listed as skipped and are not charged. For bigger sites, paste your own free key (get one in 2 minutes) and the limit goes away.

Output

Site summary (SITE_SUMMARY)

This is a real result from the default cloud run on 10 Oct 2026 (crawlee.dev, 10 pages, mobile; lists shortened to 3 entries):

{
  "site": "https://crawlee.dev",
  "discovery": {
    "sitemapsFetched": 2,
    "candidatePages": 4843,
    "pagesSelected": 10
  },
  "auditsSucceeded": 10,
  "byStrategy": {
    "mobile": {
      "averageScores": {
        "performance": 50,
        "accessibility": 95,
        "bestPractices": 96,
        "seo": 98
      },
      "medianLab": {
        "lcpMs": 4562.5,
        "tbtMs": 1554,
        "cls": 0.058,
        "fcpMs": 1968.5
      },
      "coreWebVitals": {
        "pagesWithFieldData": 10,
        "passed": 0,
        "passRatePct": 0,
        "originAssessment": {
          "lcpMs": 2754,
          "inpMs": 306,
          "cls": 0,
          "overallCategory": "AVERAGE"
        }
      },
      "worstPages": [
        {
          "url": "https://crawlee.dev/js/api/3.15/core/interface/PushErrorMessageOptions",
          "performanceScore": 37,
          "lcpMs": 13341,
          "tbtMs": 2328
        },
        {
          "url": "https://crawlee.dev/js/api/next/http-crawler/interface/DOMCrawlerOptions",
          "performanceScore": 37,
          "lcpMs": 17353,
          "tbtMs": 1965
        },
        {
          "url": "https://crawlee.dev/python/api/next/class/SelectorCacheState",
          "performanceScore": 39,
          "lcpMs": 7210,
          "tbtMs": 2910
        }
      ],
      "topOpportunities": [
        {
          "id": "cache-insight",
          "title": "Use efficient cache lifetimes",
          "pages": 10,
          "totalSavingsMs": 32450,
          "totalSavingsBytes": 0,
          "avgSavingsMs": 3245
        },
        {
          "id": "bootup-time",
          "title": "Reduce JavaScript execution time",
          "pages": 10,
          "totalSavingsMs": 16150,
          "totalSavingsBytes": 0,
          "avgSavingsMs": 1615
        },
        {
          "id": "mainthread-work-breakdown",
          "title": "Minimize main-thread work",
          "pages": 10,
          "totalSavingsMs": 16050,
          "totalSavingsBytes": 0,
          "avgSavingsMs": 1605
        }
      ],
      "recurringFailedAudits": [
        {
          "id": "errors-in-console",
          "category": "best-practices",
          "title": "Browser errors were logged to the console",
          "pages": 10
        },
        {
          "id": "color-contrast",
          "category": "accessibility",
          "title": "Background and foreground colors do not have a sufficient contrast ratio.",
          "pages": 6
        },
        {
          "id": "link-name",
          "category": "accessibility",
          "title": "Links do not have a discernible name",
          "pages": 5
        }
      ]
    }
  }
}

Per-page results (dataset)

{
  "site": "https://crawlee.dev",
  "url": "https://crawlee.dev/",
  "strategy": "mobile",
  "status": "success",
  "performanceScore": 64,
  "accessibilityScore": 95,
  "bestPracticesScore": 96,
  "seoScore": 92,
  "coreWebVitalsAssessment": "FAILED",
  "lab": {
    "lcpMs": 3520,
    "cls": 0.001,
    "tbtMs": 870,
    "fcpMs": 1810,
    "speedIndexMs": 6288,
    "ttiMs": 24148,
    "serverResponseMs": 2,
    "totalByteWeight": 2720572
  },
  "field": {
    "overallCategory": "AVERAGE",
    "isOriginFallback": false,
    "lcpMs": 2894,
    "lcpCategory": "AVERAGE",
    "inpMs": 305,
    "inpCategory": "AVERAGE",
    "cls": 0,
    "clsCategory": "FAST",
    "fcpMs": 2363,
    "fcpCategory": "AVERAGE",
    "ttfbMs": 880,
    "ttfbCategory": "AVERAGE"
  },
  "opportunities": [
    {
      "id": "cache-insight",
      "title": "Use efficient cache lifetimes",
      "savingsMs": 2100
    },
    {
      "id": "render-blocking-insight",
      "title": "Render-blocking requests",
      "savingsMs": 1200
    }
  ],
  "reportUrl": "https://pagespeed.web.dev/analysis?url=https%3A%2F%2Fcrawlee.dev%2F&form_factor=mobile"
}

Pages that fail (for example, Lighthouse can't load them) appear as "status": "error" rows. Pages over the limit appear as "status": "skipped". Both are free.

Pricing

Pay per event:

• $3.00 per 1,000 page audits ($0.003 per page and device), and
• $0.02 per site summary (once per run, only if at least one page was audited).

Example: a 10-page mobile audit costs $0.05, and 100 pages on mobile and desktop cost $0.62. Failed, skipped and retried audits are free. Set a maximum cost per run and the Actor stops starting new audits when it reaches that limit.

Related Actors

More no-login, pay-per-result tools from the same developer:

• Sitemap URL Extractor: every URL from a website's sitemaps (indexes, gzip, RSS), with lastmod and priority.
• Bulk PageSpeed Insights & Lighthouse Audit: Lighthouse scores and Core Web Vitals for a list of URLs.
• SEO Meta Tags & Structured Data Auditor: titles, descriptions, canonical, noindex, Open Graph and Schema.org rich-result checks per URL.
• Wayback Machine Snapshot Finder: archived snapshots, first/last capture and every archived URL of a domain.
• Domain Health Checker: WHOIS, DNS & SSL: domain age and expiry, DNS/SPF/DMARC, SSL certificate and HTTPS redirects.
• ATS Jobs Scraper: open jobs from Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee and Personio.

FAQ

Why only some pages and not the whole sitemap? Each PageSpeed audit takes 10–40 seconds, so a sample of 10–50 pages spread across the site usually shows the same patterns as a full audit, much faster and cheaper. Raise maxPages (with your own API key) to audit more.

The site has no sitemap. The Actor audits the URL you entered and notes that no sitemap was found in SITE_SUMMARY.discovery.

Why is the Core Web Vitals pass rate 0% while the scores look OK? The pass rate uses real-user Chrome data (CrUX), which must have good LCP, INP and CLS. Lighthouse scores are lab tests. When a page doesn't have enough traffic of its own, Google uses origin-level field data, so pages on the same site often share one verdict.

Mobile or desktop? Google ranks on mobile, so start with mobile. Use both to compare.

More tools from the same developer

• Sitemap URL Extractor: Get All URLs From Any Website: SEO specialists, agencies, developers and data/AI teams
• Bulk PageSpeed Insights Checker: Lighthouse & Core Web Vitals: SEO agencies, web developers, site owners and performance teams
• Greenhouse, Lever, Ashby & Workable Jobs Scraper (7 ATS): Job boards and aggregators, recruiters, sales teams prospecting on hiring signals, and market researchers
• Bulk WHOIS, DNS & SSL Expiry Checker: Domain Health: IT admins, agencies managing client domains, email deliverability teams, domain investors and lead-enrichment workflows