Crawl4ai
Pricing
Pay per usage
Go to Apify Store
Crawl4ai
Extract page content (markdown/HTML/text), metadata, and link stats. Uses crawl4ai.
Crawl4ai
Pricing
Pay per usage
Extract page content (markdown/HTML/text), metadata, and link stats. Uses crawl4ai.
Maximum pages to process in total.
Maximum link depth from each start URL.
Timeout per page request.
Maximum output items to push.
Only follow links within start URL domains.
Only include URLs matching these regex patterns (optional).
Retry failed pages up to this count.
Base backoff in seconds, doubled each retry.
Global rate limit. Set 0 for unlimited.
Remove navigation-heavy lines and normalize whitespace.
Include unmodified content output in a separate field.
Truncate content to this length (0 = unlimited).
Length of the content excerpt for quick previews.
Ignore text blocks with fewer words (0 = off). Reduces noise from empty or stub pages.
CSS selector for infinite-scroll container (e.g. #feed). When set, the crawler scrolls to load more content before extraction.
Max scroll steps when virtual scroll is enabled.
Page load strategy: domcontentloaded (fast), load (full load), or networkidle (SPA/slow sites).
Extra delay in seconds after load before capturing HTML. Use for slow/SPA sites.
CSS selector to wait for before extraction (e.g. .article-body or #main). Use css: or js: prefix for advanced conditions.
Max seconds to wait for Wait For Selector. Ignored if Wait For Selector is empty.
Extract only content inside this CSS selector (e.g. main, .content, #article).
full = extract content; discover_only = only URLs and links (no content, fast).
