Firecrawl Website Crawler
Pricing
from $0.01 / 1,000 results
Firecrawl Website Crawler
Enhanced Website Crawling with Superior JS Rendering Enhanced website crawler using Firecrawl's Crawl API for superior JavaScript rendering, smart rate limiting, anti-bot bypass, and clean markdown extraction.
Pricing
from $0.01 / 1,000 results
Rating
0.0
(0)
Developer
The Howlers
Maintained by CommunityActor stats
1
Bookmarked
59
Total users
5
Monthly active users
2 days ago
Last modified
Categories
Share
Enhanced website crawler using Firecrawl's Crawl API for superior JS rendering, smart rate limiting, anti-bot bypass, and clean markdown extraction.
BYOK (Bring Your Own Key): you provide your own API credentials.
Before You Start
This actor requires your own API credentials to fetch real data.
Where to get your key: Your Firecrawl API key. Get one at firecrawl.dev
Quick Start
Run with real data
Input Parameters
| Parameter | Type | Default | Required | Description |
|---|---|---|---|---|
url | string | - | Yes* | The website URL to crawl (e.g., https://example.com) |
maxPages | integer | 100 | No | Maximum number of pages to crawl |
maxDepth | integer | 5 | No | Maximum crawl depth from the starting URL |
includeSubdomains | boolean | false | No | Crawl subdomains of the main domain |
excludePatterns | array | - | No | URL patterns to exclude from crawling (regex supported) |
includePatterns | array | - | No | Only crawl URLs matching these patterns (regex supported) |
outputFormat | string | "markdown" | No | Format for extracted content |
includeScreenshots | boolean | false | No | Capture screenshots of each page |
waitForSelector | string | - | No | CSS selector to wait for before extracting content (for JS-heavy sites) |
country | string | - | No | Country code for geo-targeted crawling (e.g., 'US', 'GB') |
firecrawlApiKey | string | - | Yes* | Your Firecrawl API key. Get one at firecrawl.dev |
webhookUrl | string | - | No | URL to receive results via webhook when complete |
Troubleshooting
"API key is required"
- Add your API key in the input
"API error 403" or "Unauthorized"
Your API key is invalid, expired, or doesn't have access to this specific API endpoint. Double-check your key and account permissions.
"API error 429" or "Rate limit"
Too many requests. Wait a minute and try again, or reduce the number of items per run.
No results or empty dataset
Check the run log for error messages. Common causes:
- Invalid input format (check the examples above)
- API key without proper permissions
- The target data doesn't exist or is too small to track
How do I test without an API key?
Run the actor without a firecrawlApiKey, or set "demoMode": true. It returns realistic sample results labeled with demo: true and a _notice field so you can inspect the exact output shape before connecting your own key. Demo runs make no external API calls and charge no task events.
Built by John Rippy | Actor Arsenal
