Pricing

from $0.01 / 1,000 results

Website Content Crawler

Crawl websites for SEO audits. Extracts HTML, title, meta tags, headings, links, & text content from pages. Automatic sitemap detection & parsing Extracts metadata (title, description, OG tags) Heading structure (H1, H2, H3) Internal & external link analysis Image extraction w/alt text Word count

Pricing

from $0.01 / 1,000 results

Rating

0.0

(0)

Developer

The Howlers

Actor stats

Bookmarked

133

Total users

Monthly active users

7 days ago

Last modified

Website Crawler - SEO Audit Crawler with Markdown Extraction

Fast, reliable website crawler built for SEO audits and AI/LLM content analysis. Auto-discovers sitemaps, extracts metadata, headings, and LLM-ready markdown content from every page.

BYOK (Bring Your Own Key) -- you provide your own API credentials.

Before You Start

This actor requires your own API credentials to fetch real data.

Where to get your key: Your Firecrawl API key for web crawling. When provided, Firecrawl is used as the primary crawler (faster, handles JS rendering). Falls back to Playwright if not provided or if Firecrawl fails.

Quick Start

Run with real data

{
  "startUrls": "https://example.com",
  "maxCrawlPages": 25,
  "maxCrawlDepth": 2,
  "crawlSitemap": true,
  "firecrawlApiKey": "YOUR_API_KEY_HERE",
  "useFirecrawl": true
}

Input Parameters

Parameter	Type	Default	Required	Description
`startUrls`	array	-	No	URLs to start crawling from
`maxCrawlPages`	integer	`25`	No	Maximum number of pages to crawl
`maxCrawlDepth`	integer	`2`	No	Maximum link depth to follow
`crawlSitemap`	boolean	`true`	No	Try to find and use sitemap for URL discovery
`firecrawlApiKey`	string	-	Yes*	Your Firecrawl API key for web crawling. When provided, Firecrawl is used as the primary crawler (faster, handles JS rendering). Falls back to Playwright if not provided or if Firecrawl fails.
`useFirecrawl`	boolean	`true`	No	Use Firecrawl as the primary crawling method. Requires Firecrawl API key. Falls back to Playwright if disabled or if Firecrawl fails.
`webhookUrl`	string	-	No	URL to POST results when scraping completes (Zapier, Make, n8n, custom endpoint)

Pricing

This actor uses pay-per-event billing:

Event	Description	Price
Page Crawled	Per page crawled with full HTML, text, and markdown extraction	$0.05
Sitemap Discovered	Per sitemap discovered and parsed for URL extraction	$0.05

Troubleshooting

"API key is required"

Add your API key in the input

"API error 403" or "Unauthorized"

Your API key is invalid, expired, or doesn't have access to this specific API endpoint. Double-check your key and account permissions.

"API error 429" or "Rate limit"

Too many requests. Wait a minute and try again, or reduce the number of items per run.

No results or empty dataset

Check the run log for error messages. Common causes:

Invalid input format (check the examples above)
API key without proper permissions
The target data doesn't exist or is too small to track

How do I test without an API key?

Built by John Rippy | Actor Arsenal

Website Analyzer Crawler

quarterly_lettuce/website-analyzer-crawler

A powerful web crawler that analyzes websites and extracts comprehensive SEO data including meta tags, headings structure, word count, internal/external links, and images.

Abhishek Kumar Giri

Bulk SEO Data Extractor

thirdwatch/seo-data-extractor

Extract every on-page SEO signal from any URL: title, meta tags, canonical, OG/Twitter cards, JSON-LD schema, heading hierarchy, alt-text gaps, internal/external link counts, word count, text-to-HTML ratio.

Thirdwatch

SEO Audit Scraper

actorfordge/seo-audit-scraper

Crawl any website and run an on-page SEO audit. This Actor checks page titles, meta descriptions, H1/H2 headings, word count, canonical URLs, image alt text, internal links, external links, and gives each page an SEO score with clear issues found.

Will E

Website Content Crawler — Text, Titles & Metadata

hipersoft/website-content-crawler

Extract clean, readable content from any list of websites: page title, meta description, headings, main body text, word count and link/image counts. Optional same-domain crawl. Bulk-ready, no browser, no login. Great for LLM/RAG ingestion, content audits and research.

hiper soft

Sitemap Content Extractor

darknezz/sitemap-content-extractor

Crawl any website sitemap.xml and extract structured content from each page. Full-text extraction, metadata, headings, and word counts for SEO audits and content inventories.

Oaida Adrian

Sitemap URL & SEO Metadata Extractor

happitap/sitemap-seo-extractor

Crawl any XML sitemap and instantly extract comprehensive SEO metadata (Title, Description, H1s, OG Tags, Canonical) for every page

HappiTap

SEO Data Extractor - On-Page Tags for Any URL

dltik/seo-data-extractor

Extract on-page SEO data from any list of URLs: title, meta description, canonical, robots, H1/H2 headings, Open Graph & Twitter Card tags, lang, viewport and word count. Pure HTTP, no browser — fast and cheap for bulk audits.

Walid

Website Meta Tags & Open Graph Scraper

fit_melon/website-meta-tags-scraper

Extract SEO meta tags from any list of URLs: title, meta description, canonical, robots, Open Graph (og:image, og:title), Twitter Cards, favicons, language. Clean JSON for SEO audits and link previews. Free.

D N

Universal Website Meta Scraper — SEO & Links Analysis

scrapepilot/universal-website-meta-scraper----seo-links-analysis

Extract meta data from any website instantly. Get title, description, headings, links, images, OG tags & status code. Perfect for SEO analysis, lead gen, and auditing. No coding required.

Scrape Pilot

Website Content Miner

seeb/website-content-miner

Extract clean website content at scale: page titles, meta descriptions, H1-H3 headings, readable main text, and URLs. Includes smart noise removal, Readability fallback, optional internal crawling, and structured output for SEO audits, AI datasets, research, and automation.