Website Content Crawler Lite
Pricing
from $0.10 / 1,000 page extracteds
Crawl public website pages and extract clean text, Markdown, metadata, and links for AI, SEO, and monitoring workflows.
Pricing
from $0.10 / 1,000 page extracteds
Rating
0.0
(0)
Developer
Hanna Nosova
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
12 hours ago
Last modified
Categories
Share
Maximum number of pages to fetch in this run, including skipped pages.
How many link levels to follow from each start URL. Use 0 to extract only the start pages.
When enabled, discovered links outside the start URL's domain are reported but not crawled.
Optional glob patterns. If set, only URLs matching at least one pattern are crawled. Example: https://example.com/docs/**
[]Optional glob patterns for URLs to skip. Example: /login
[ "**/login**", "**/signup**"]Choose whether each successful page includes Markdown, plain text only, or cleaned HTML in addition to metadata and links.
Check robots.txt before crawling a page and skip disallowed URLs.