Pricing

Pay per usage

Go to Store

Crawl4AI

Try for free

Developed by

Jan Buchar

Wraps the Crawl4AI open-source library for retrieving text content from websites.

0.0 (0)

Pricing

Pay per usage

Monthly users

Runs succeeded

81%

Last modified

11 days ago

Start URLs

startUrlsarrayRequired

URLs to start with

Maximum crawl depth

maxCrawlDepthintegerOptional

How many levels of links should the crawler follow. The Start URLs have a depth of 0, all links found there have a depth of 1, and so on.

Default value of this property is 2

Proxy configuration

proxyConfigurationobjectOptional

Select proxies to be used by your crawler.

Use magic mode

magicModebooleanOptional

Masks browser automation signals, Simulates human-like behavior, Overrides navigator properties, Handles cookie consent popups, Manages browser fingerprinting, Randomizes timing patterns

Default value of this property is true

Save screenshots

saveScreenshotsbooleanOptional

Should the crawler save screenshots?

Default value of this property is false

CSS selector

cssSelectorstringOptional

Ensures only a part of the website is processed

Wait for

waitForstringOptional

CSS selector or JS function to wait for before processing a page

Javascript code to execute on each page

jsCodestringOptional

Function executed for each request

Extraction strategy

extractionStrategyEnumOptional

Specifies how data is extracted from each page

Value options:

"llm": string"jsonCss": string"cosine": string

LLM provider

llmProviderstringOptional

Specifies the LLM used to extract data

API token

llmApiTokenstringOptional

Token used for authentication with the LLM provider

Instruction

llmInstructionstringOptional

Instructions (prompt) for the LLM

JsonCss schema

jsonCssSchemastringOptional

Description of the data to be extracted

Semantic filter

cosineSemanticFilterstringOptional

Specifies the semantic filter

Word count threshold

cosineWordCountThresholdintegerOptional

Specifies the word count threshold

Linkage method

cosineLinkageMethodstringOptional

Specifies the linkage method

Top K

cosineTopKintegerOptional

Specifies top K

Model name

cosineModelNamestringOptional

Specifies the model

Debug logging

debugLogbooleanOptional

Print debug-level log messages

Default value of this property is false

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.

AI Web Scraper - Powered by Crawl4AI

raizen/ai-web-scraper

A blazing-fast AI web scraper powered by Crawl4AI. Perfect for LLMs, AI agents, AI automation, model training, sentiment analysis, and content generation. Supports deep crawling, multiple extraction strategies and flexible output (Markdown/JSON). Seamlessly integrates with Make.com, n8n, and Zapier.

Raizen Technology

AI-Powered Web Content & Link Extractor

scrapercoder/ai-powered-web-content-link-extractor

Crawls websites to extract clean, structured content for AI/LLM use, ideal for training datasets, knowledge bases, and RAG systems. Json output includes: * text: Normalized page content * links: Extracted sub-URLs

wallnut.ai

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

43.6k

Monolith

netmilk/monolith

A data hoarder’s dream come true: bundle any web page into a single HTML file. This Actor is a wrapper of the popular open-source library.

Adam Kliment

AI Website Content Markdown Scraper

quaking_pail/ai-website-content-markdown-scraper

This Apify Actor, "Website Content Crawler with Markdown Extraction," is designed to perform a comprehensive crawl of specified websites, extract their text content, convert it into Markdown format, and store it in a structured dataset. The extracted content is suitable for feeding LLMs.

AI_Builder

473

HTML/Web Media Scraper

aweworkz/html-web-media-scraper

Extracts various media files, such as images, videos, audio, and other related media elements, from multiple websites. It then provides the corresponding descriptions or the alt="" content. You may need to use proxies to run this actor for some websites with bot blocking features.

aweworkz

140

Fast Website Content Crawler

6sigmag/fast-website-content-crawler

A high-performance web scraper that rapidly extracts and analyzes content from multiple websites simultaneously. Perfect for competitive research, content aggregation, and website structure analysis.

David Deng

502

Website Content Extractor

fastidious_drawer/website-content-extractor

This extractor lets you extract content from any website with a single or multiple URLs. Use selectors to choose specific sections like the body and exclude elements like headers or navigation. It also extracts images and links, providing data in JSON and DataTable formats for easy processing.