Pricing

$4.20 / 1,000 page analyzeds

Try for free

Go to Apify Store

Sitemap & On-Page Crawl

Try for free

Discovers a site's sitemap via robots.txt, crawls up to maxPages pages with CheerioCrawler, and returns per-page SEO/on-page metrics.

Pricing

$4.20 / 1,000 page analyzeds

Rating

0.0

(0)

Developer

Mark

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

What it does

Discovers a site's sitemap (via robots.txt, falling back to /sitemap.xml or /sitemap.txt), respects robots.txt disallow rules, then crawls up to maxPages of the listed pages with Crawlee's CheerioCrawler and returns one on-page SEO/metrics row per page.

Input

Field	Type	Required	Description
`url`	string	yes	Any URL on the target site — its origin's robots.txt/sitemap is used.
`maxPages`	integer	no	Max pages to crawl (default 100).

Output example

One row per crawled page in the default dataset:

{
  "url": "https://crawlee.dev/blog",
  "status": 200,
  "title": "Crawlee Blog - learn how to build better scrapers | Crawlee for JavaScript · Build reliable crawlers. Fast.",
  "metaDescription": "Guides and tutorials on using Crawlee, the most reliable open-source web scraping and browser automation library for JavaScript and Node.js developers.",
  "h1": null,
  "canonical": "https://crawlee.dev/blog",
  "wordCount": 1370,
  "internalLinks": 82,
  "externalLinks": 64,
  "imagesWithoutAlt": 0,
  "loadMs": 471
}

Pricing

Pay-per-event (PPE). One page-analyzed event is charged per dataset row via Actor.charge({ eventName: 'page-analyzed' }) — never charged for a page that errors before a row is pushed. Configure the event at $0.0042/page-analyzed in the Apify Console pricing step (policy/RULES.md rule 3: ~10% under apify/sitemap-extractor's $0.0005/item baseline scaled to a full on-page pass, and 10x under the $0.04/page audit-tool alternative).

Limits

Free-plan compute only (policy/RULES.md rule 4). No residential proxy needed. JS-rendered content is not executed (Cheerio, not a browser) — pages that inject title/meta/h1 via client-side JS will show blank/null for those fields. No personal data in output (rule 1).

Website SEO Spider — On-Page Crawl Export

khadinakbar/website-seo-spider

Crawl a public website and export one on-page SEO row per URL (title, meta, H1, canonical, indexability, inlinks, issue flags). Use for site crawls and scheduled audits. Pair with broken-link-checker or complete-seo-audit. Charged $0.01 per page crawled.

Khadin Akbar

Website H1-H6 Heading Checker - SEO Structure Audit

smart-digital/website-h1-h6-headings-checker

Check H1-H6 headings on one page or crawl a whole site. Returns every heading by level, a 0-100 structure score per page, and the exact issues found - missing H1, duplicate H1, skipped levels, headings too short or long - each with a severity and a fix, plus a site-wide summary.

My Smart Digital

DomainForge LLM Dataset Builder

actorify/domainforge-llm-dataset-builder

Crawls websites and transforms web content into clean, structured datasets optimized for LLM fine-tuning, RAG applications, and knowledge base construction

fanio zilla

Website Contact & Social Discovery Crawler

competent_clarinet/website-contacts-scraper

High-throughput crawler that extracts emails, phone numbers, and social media profiles from websites using HTTP-first Crawlee crawling with Selectolax parsing and Playwright SPA fallback.

Man Mohit verma

Broken Link Checker & Site Auditor

andok/broken-links-checker

Crawl websites to detect 404 broken links and missing resources. Essential for maintaining technical SEO and user experience.

Andok

Website Content Scraper

qaseemiqbal/website-content-scraper

Extract clean Markdown, plain text, linked files, and RAG-ready chunks from websites, documentation, help centers, knowledge bases, and authenticated portals. Preserve structure, metadata, URLs, and crawl context for AI search, training, and retrieval workflows.

Muhammad Qaseem Iqbal

Technical On-Page SEO & Meta Auditor

gp005/seo-onpage-auditor

Audit any website's on-page SEO: titles, meta descriptions, H1 tags, canonical URLs, hreflang, Open Graph, Twitter Cards, JSON-LD schema, broken links, image alt text, and robots compliance. Single polite crawl, first-party domains only.

Geo

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Stas Persiianenko

SEO Audit API — On-Page + Core Web Vitals, Bulk & Scored

fetchbase/website-seo-audit

SEO audit with 0–100 score, real Core Web Vitals (LCP, CLS, FCP) measured in a live browser, and prioritized issues each with a concrete fix. Meta, headings, canonical, OG, JSON-LD, alt text, robots.txt, sitemap, broken links. Single pages or site crawls. Charged only per audited page.