Pricing

from $0.01 / 1,000 results

Firecrawl Agent - Web Crawler

Advanced web crawling with Firecrawl. Extract clean markdown, handle JavaScript sites & manage large-scale crawls with built-in rate limiting & error handling.

Pricing

from $0.01 / 1,000 results

Rating

0.0

(0)

Developer

The Howlers

Actor stats

Bookmarked

Total users

Monthly active users

a month ago

Last modified

Categories

Developer tools

Automation

Agents

Starting URL

url

Optional

The URL where the agent should start its extraction

Type:string

Extraction Objective

objective

Optional

Describe in natural language what data you want to extract. The AI will autonomously navigate and find it.

Type:string

Output Schema (Optional)

schema

Optional

JSON schema defining the structure of extracted data. Leave empty for automatic extraction.

Type:object

Max Navigation Steps

maxSteps

Optional

Maximum number of pages/actions the agent can take

Type:integer

Minimum:1

Maximum:50

Default:10

Allow Backtracking

allowBacktracking

Optional

Allow the agent to go back and try different paths if stuck

Type:boolean

Default:true

Firecrawl API Key

firecrawlApiKey

Optional

Your Firecrawl API key (get it at firecrawl.dev)

Type:string

Demo Mode

demoMode

Optional

Run with sample data (no API key required). Disable and provide a Firecrawl API key for real extraction.

Type:boolean

Default:true

Webhook URL

webhookUrl

Optional

URL to send extraction results via POST request. Supports Zapier, Make, n8n, and custom webhooks.

Type:string

Webhook Platform

webhookPlatform

Optional

Select your webhook platform for optimized payload formatting

Type:string

Default:custom

Options:

zapiermaken8ncustom

Custom Webhook Headers

webhookHeaders

Optional

Additional HTTP headers to include in webhook requests (e.g., authorization tokens)

Type:object

Cloudflare Web Scraper

ecomscrape/cloudflare-web-scraper

Advanced web scraper designed to extract data from Cloudflare-protected websites with CAPTCHA bypass, proxy rotation, and JavaScript execution capabilities.

ecomscrape

745

3.3

Camoufox Scraper

apify/camoufox-scraper

Crawls websites with stealthy Camoufox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

306

5.0

Actor Benchmark

apify/actor-benchmark

Compares various builds of the same actor to measure how they perform on the same input

Apify

Sitemap Extractor

apify/sitemap-extractor

This Apify Actor extracts all URLs from a website's sitemaps and checks their status codes via lightweight HTTP requests. It provides a clean list of valid links, acting as an ideal pre-processor to ensure your larger crawling projects target only active URLs.

Apify

129

5.0

AI Web Scraper

apify/ai-web-scraper

AI-first web scraper that extracts structured data from any website using natural-language prompts. No programming knowledge required. No hard-coded logic that breaks when a website changes.

Apify

7.2K

3.5

JSDOM Scraper

apify/jsdom-scraper

Parses the HTML using the JSDOM library, providing the same DOM API as browsers do (e.g. `window`). It is able to process client-side JavaScript without using a real browser. Performance-wise, it stands somewhere between the Cheerio Scraper and the browser scrapers.

Apify

153

4.3

Web Scraper

apify/web-scraper

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

118K

4.4

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

14K

5.0

Page Scraping Analyzer

apify/page-analyzer

Performs analysis of a webpage to figure out the best way how to scrape its data. Provide a URL and data points to find and get back a detailed dashboard showing how the data can be scraped. Works with initial and rendered HTML, JavaScript variables and dynamically loaded data.

Apify

1.3K

4.7

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

17K

4.5