Pricing

from $0.01 / 1,000 results

Firecrawl Website Crawler

Enhanced Website Crawling with Superior JS Rendering Enhanced website crawler using Firecrawl's Crawl API for superior JavaScript rendering, smart rate limiting, anti-bot bypass, and clean markdown extraction.

Pricing

from $0.01 / 1,000 results

Rating

0.0

(0)

Developer

The Howlers

Actor stats

Bookmarked

Total users

Monthly active users

5 days ago

Last modified

Before You Start

This actor requires your own API credentials to fetch real data.

Where to get your key: Your Firecrawl API key. Get one at firecrawl.dev

You can test with Demo Mode first (free, no key needed) to see the output format before committing.

Quick Start

Test with Demo Mode (free, no API key needed)

{
  "demoMode": true,
  "url": "https://example.com",
  "includeSubdomains": [
    "example.com"
  ]
}

Run with real data

{
  "demoMode": false,
  "url": "https://example.com",
  "maxPages": 100,
  "maxDepth": 5,
  "includeSubdomains": false,
  "outputFormat": "markdown",
  "includeScreenshots": false,
  "firecrawlApiKey": "YOUR_API_KEY_HERE"
}

Input Parameters

Parameter	Type	Default	Required	Description
`url`	string	-	Yes*	The website URL to crawl (e.g., https://example.com)
`maxPages`	integer	`100`	No	Maximum number of pages to crawl
`maxDepth`	integer	`5`	No	Maximum crawl depth from the starting URL
`includeSubdomains`	boolean	`false`	No	Crawl subdomains of the main domain
`excludePatterns`	array	-	No	URL patterns to exclude from crawling (regex supported)
`includePatterns`	array	-	No	Only crawl URLs matching these patterns (regex supported)
`outputFormat`	string	`"markdown"`	No	Format for extracted content
`includeScreenshots`	boolean	`false`	No	Capture screenshots of each page
`waitForSelector`	string	-	No	CSS selector to wait for before extracting content (for JS-heavy sites)
`country`	string	-	No	Country code for geo-targeted crawling (e.g., 'US', 'GB')
`firecrawlApiKey`	string	-	Yes*	Your Firecrawl API key. Get one at firecrawl.dev
`webhookUrl`	string	-	No	URL to receive results via webhook when complete
`demoMode`	boolean	`true`	No	Run in demo mode with sample data (no API key required). Set to false when using your own Firecrawl API key.

*Required when Demo Mode is off.

Troubleshooting

"API key is required"

You have Demo Mode turned off but didn't provide an API key. Either:

Turn Demo Mode on to test with sample data
Add your API key in the input

"API error 403" or "Unauthorized"

Your API key is invalid, expired, or doesn't have access to this specific API endpoint. Double-check your key and account permissions.

"API error 429" or "Rate limit"

Too many requests. Wait a minute and try again, or reduce the number of items per run.

No results or empty dataset

Check the run log for error messages. Common causes:

Invalid input format (check the examples above)
API key without proper permissions
The target data doesn't exist or is too small to track

How do I test without an API key?

Enable Demo Mode in the input. This returns realistic sample data so you can verify the output format works for your workflow.

Built by John Rippy | Actor Arsenal

Firecrawl Agent - Web Crawler

alizarin_refrigerator-owner/firecrawl-agent

Advanced web crawling with Firecrawl. Extract clean markdown, handle JavaScript sites & manage large-scale crawls with built-in rate limiting & error handling.

The Howlers

AI Web Crawler

hounderd/ai-web-crawler

Crawl websites and extract clean, LLM-ready markdown content with stealth browser rendering, anti-bot hardening, smart content filtering, and structured metadata extraction. Built for RAG pipelines, AI agents, and data workflows.

Hounderd

Clean Web Scraper - Markdown for AI via Firecrawl

clearpath/web-to-markdown

Convert any website to clean, LLM-optimized markdown using Firecrawl. Perfect for RAG pipelines, AI training data, and knowledge bases. No login required, 25% cheaper than Firecrawl direct. Batch process hundreds of URLs. Supports PDF/DOCX. Pay only $0.004 per page - no monthly fees.

ClearPath

Firecrawl Pro Advanced Web Scraping Full Firecrawl Features

alizarin_refrigerator-owner/firecrawl-pro-advanced-web-scraping-full-firecrawl-features

Professional scraping using Firecrawl's complete feature set / exposes all Firecrawl capabilities Markdown/HTML Content Filtering Include/Exclude Screenshots Stealth Mode Fast Mode Caching Location Proxies JSON Prompt-Based Branding Link Extraction Image Extraction Autonomous Multi-Page Discovery

The Howlers

Advanced Website Crawling Actor

techforce.global/advanced-website-crawling-actor

A fast and reliable scraper for any website that extracts clean HTML, Markdown, and text content. Provides clean, structured data with support for dynamic rendering, recursive sitemap discovery, SSL bypass, and easy API integration for your applications.

Techforce Global

5.0

🕷️ Website Crawler — Full-Site Scraping for AI

nexgendata/website-content-crawler

Crawl entire websites for clean text, markdown or HTML. Perfect for RAG pipelines, AI training & content analysis. Handles JS-rendered pages. Alternative to Firecrawl & Jina. Pay per page.

Stephan Corbeil

Firecrawl Enrich - Data Enrichment

alizarin_refrigerator-owner/fire-enrich

Enrich business data with Firecrawl. Get company info, contact details, social profiles & website data. Batch process leads for sales & marketing.

The Howlers

Stealth Website Crawler

nocturne/stealth-website-crawler

Crawl websites protected by Cloudflare, DataDome, and other anti-bot systems. Extract clean text or markdown for AI/LLM pipelines. Like Website Content Crawler, but for sites that block you.

Nocturne

AI Website Content Crawler

ilborso/ai-website-content-crawler

A super fast website crawler for AI training

Fabio Borsotti

Universal AI Web Scraper

stanvanrooy6/universal-ai-web-scraper

Turn any website into an API. Extract structured data using plain English. Features anti-bot bypass, dynamic rendering, and web search. No coding needed.