Crawl4AI MCP server
Struggling to extract clean web content for your AI applications? Crawl4AI MCP server gives your AI agents direct access to smart web scraping with automatic markdown conversion, link following, and LLM-optimized extraction.
POWERING THE WORLD'S TOP DATA-DRIVEN TEAMS
What is Crawl4AI MCP server?
Crawl4AI MCP server connects AI agents with Crawl4AI scraper via the Model Context Protocol. It fetches clean Markdown output, HTML content, website metadata, structured extraction data, and more.
Advanced extraction capabilities
Extract content using CSS selectors, XPath queries, or LLM-based methods for maximum precision and flexibility.
Clean markdown output generation
Produces structured markdown format optimized for RAG pipelines and direct ingestion into large language models.
Intelligent automation features
Automatically follows links throughout websites and retries failed requests to ensure comprehensive data collection with minimal manual intervention.
One token to rule them allSingle Apify API key unlocks 6,000+ Actors and MCP servers. One account, one invoice and transparent usage tracking.
Dynamic tool discoveryIs Crawl4AI MCP server not enough? Agents can search, evaluate, and add other Apify tools based on workflow needs.
Smart prompts and resourcesPre-built prompts and resources help agents understand tool capabilities and use cases.
Apify API Token
Get your personal API token from the Integrations section in Apify Console. This token will be used to authorize the MCP server to run Actors on your behalf.
MCP client
An AI agent or client that supports MCP. This could be Anthropic Claude for Desktop, a VS Code extension with MCP support, Apify’s web-based Tester MCP Client, or any custom client implementation. You can also check supported MCP clients in official documentation.
How to set up Crawl4AI MCP server
If you have an MCP client that supports streamable HTTP, connect using OAuth authentication by providing Crawl4AI MCP server URL you can find below. You'll be redirected to your browser to sign in to your Apify account and approve the connection.
1{2 "mcpServers": {3 "apify": {4 "url": "https://mcp.apify.com/?actors=janbuchar/crawl4ai"5 }6 }7}If you don't have an MCP client that supports streamable HTTP, you can still set up Apify's Crawl4AI MCP server. For Claude for Desktop, enable MCP servers via the Developer settings. Then, you need to add an entry for the Apify MCP server. For instance, in Claude’s config file, under mcpServers, add an entry like:
{ "mcpServers": { "actors-mcp-server": { "command": "npx", "args": [ "-y", "@apify/actors-mcp-server", "--actors", "janbuchar/crawl4ai" ], "env": { "APIFY_TOKEN": "YOUR_APIFY_TOKEN" } } } }MCP server configuration
Configure your own Apify MCP server exactly as you need with Actors and tools for easy integration with MCP clients.
Sample dataset
Here's a sample of the data you can extract with our Crawl4AI MCP server including clean web content with automatic markdown conversion, structured data through LLM-optimized extraction, linked content through intelligent link following, and smart web scraping capabilities, and much more without the limitation of traditional platforms' official APIs.
[{ "url": "https://docs.crawl4ai.com/", "markdown": "https://api.apify.com/v2/key-value-stores/m1Sqnke1KWM0AI8co/records/content_4242424242.md", "html": "https://api.apify.com/v2/key-value-stores/m1Sqnke1KWM0AI8co/records/content_4242424242.html", "metadata": { "title": "Home - Crawl4AI Documentation (v0.5.x)", "description": "🚀🤖 Crawl4AI, Open-source LLM-Friendly Web Crawler & Scraper" }},{ "url": "https://docs.crawl4ai.com/advanced/ssl-certificate/", "markdown": "https://api.apify.com/v2/key-value-stores/m1Sqnke1KWM0AI8co/records/content_4242424242.md", "html": "https://api.apify.com/v2/key-value-stores/m1Sqnke1KWM0AI8co/records/content_4242424242.html", "metadata": { "title": "SSL Certificate - Crawl4AI Documentation (v0.5.x)", "description": "🚀🤖 Crawl4AI, Open-source LLM-Friendly Web Crawler & Scraper" }},// ...]
The Crawl4AI MCP server is a tool that enables AI agents and applications to extract clean web content, markdown-formatted text, structured data, and linked resources from websites through smart web scraping with automatic conversion and LLM-optimized processing. It provides programmatic access to comprehensive data beyond the limitations of traditional web scraping's official API.
To use the Crawl4AI MCP server, create an Apify account and get your API token. Then configure the server in your MCP-compatible client using either OAuth authentication or by adding the configuration with your token.
No, you don't need Crawl4AI API key to use our Crawl4AI MCP server. The server operates through Apify's infrastructure and only requires an Apify account with API token for authentication and access to the service.
You can access comprehensive business data including place names, addresses, phone numbers, websites, ratings, review counts, opening hours, photos, and detailed reviews. The server also extracts categories, pricing information, amenities, and additional business details from Google Maps listings.
The Crawl4AI MCP server has no specific rate limits imposed by the service itself, but usage is subject to your Apify account's resource limits and pricing plan.