Firecrawl Agent - AI Autonomous Web Data Extraction

Extract data from any website using plain English. No code, no selectors, no scraping knowledge needed. Just describe what you want and the AI agent finds it.

Before You Start (Read This First)

Step	What to Do	Time
1	Click Start with default settings to see demo output	10 seconds
2	Get a free Firecrawl API key at firecrawl.dev	2 minutes
3	Uncheck Demo Mode, paste your API key, enter a URL + objective	1 minute

That's it. If you skip Step 2, the actor returns sample data so you can preview the output format risk-free.

How It Works (3 Inputs)

You only need 3 things for real extraction:

1. firecrawlApiKey  ->  Your key from firecrawl.dev (free tier: 500 credits/month)
2. url              ->  The website to extract from (e.g. https://competitor.com)
3. objective        ->  What you want, in plain English (e.g. "Find all pricing plans")

The AI agent reads the page, follows links, and returns structured data. No CSS selectors. No XPath. No code.

Quick Start Examples

1. Try Demo Mode First (No API Key Needed)

{
  "demoMode": true
}

Returns realistic sample data so you can see the output format before paying anything.

2. Extract Competitor Pricing

{
  "firecrawlApiKey": "fc-YOUR-KEY-HERE",
  "url": "https://competitor.com",
  "objective": "Find all pricing plans with monthly price, annual price, and included features",
  "demoMode": false
}

3. Research a Company's Leadership Team

{
  "firecrawlApiKey": "fc-YOUR-KEY-HERE",
  "url": "https://company.com",
  "objective": "Find the leadership team, their titles, and LinkedIn profiles",
  "maxSteps": 15,
  "demoMode": false
}

4. Extract Job Listings

{
  "firecrawlApiKey": "fc-YOUR-KEY-HERE",
  "url": "https://company.com/careers",
  "objective": "Get all software engineering job openings with title, location, and salary if shown",
  "maxSteps": 15,
  "demoMode": false
}

5. Structured Output with JSON Schema

{
  "firecrawlApiKey": "fc-YOUR-KEY-HERE",
  "url": "https://store.com/products",
  "objective": "Extract all products with name, price, and availability",
  "schema": {
    "type": "array",
    "items": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "price": { "type": "number" },
        "inStock": { "type": "boolean" }
      }
    }
  },
  "demoMode": false
}

All Input Parameters

Parameter	Type	Required?	Default	Description
demoMode	boolean	No	true	Returns sample data. Uncheck this for real extraction.
url	string	Yes (if not demo)	-	The website URL to extract data from
objective	string	Yes (if not demo)	-	Describe what you want in plain English
firecrawlApiKey	string	Yes (if not demo)	-	Your API key from firecrawl.dev
schema	object	No	-	Optional JSON schema for strictly structured output
maxSteps	number	No	10	How many pages the agent can visit (increase for complex sites)
allowBacktracking	boolean	No	true	Let the agent try different paths if the first one fails
webhookUrl	string	No	-	URL to POST results to (Zapier, Make, n8n, etc.)
webhookPlatform	string	No	custom	zapier, make, n8n, or custom

Where to Get a Firecrawl API Key

1. Go to firecrawl.dev
2. Click Sign Up (free tier includes 500 credits/month)
3. Go to Dashboard > API Keys
4. Click Create API Key and copy it
5. Paste it into the firecrawlApiKey field

Output Format

Every run produces a dataset with this structure:

{
  "success": true,
  "url": "https://competitor.com",
  "objective": "Find all pricing plans with features and costs",
  "data": {
    "plans": [
      {
        "name": "Starter",
        "monthlyPrice": 29,
        "features": ["5 users", "10GB storage", "Email support"]
      },
      {
        "name": "Pro",
        "monthlyPrice": 99,
        "features": ["Unlimited users", "100GB storage", "Priority support"]
      }
    ]
  },
  "stepsUsed": 4,
  "pagesVisited": [
    "https://competitor.com",
    "https://competitor.com/pricing"
  ],
  "extractedAt": "2026-01-28T10:30:00.000Z"
}

If something goes wrong, success will be false with an error field explaining what happened.

Pricing (Pay-Per-Event)

Event	Price
Each extraction completed	$0.05
Demo mode	Free

Example costs:

• 1 competitor analysis: $0.05
• 10 competitor analyses: $0.50
• 100 company profiles: $5.00

Firecrawl API usage is billed separately through your Firecrawl account (~$0.01-0.05 per extraction in Firecrawl credits).

Troubleshooting

"I'm seeing demo/sample data"

You have Demo Mode enabled (it's on by default). Uncheck Demo Mode and provide your Firecrawl API key + URL.

"Missing required input: firecrawlApiKey"

You need a Firecrawl API key. Get one free at firecrawl.dev (see instructions above).

Agent returns empty or incomplete data

• Be more specific in your objective. "Find pricing" is vague. "Find all pricing plans with monthly price, annual price, and features included in each plan" is specific.
• Increase maxSteps to 15-20 for complex sites with deep navigation.
• Enable allowBacktracking (on by default) so the agent can try alternative paths.

Agent times out

• Lower maxSteps or simplify your objective.
• Break complex extractions into multiple runs (e.g., pricing in one run, team in another).

Schema validation errors

• Make your schema flexible — use "type": ["string", "null"] for fields that might be missing.
• Simplify your schema to fewer fields.

"Invalid API key" error

• Copy your key from firecrawl.dev dashboard — make sure there are no extra spaces.
• Check that your Firecrawl account is active and has credits remaining.

Webhook Integration

Send results to Zapier, Make.com, n8n, or any webhook endpoint automatically:

{
  "firecrawlApiKey": "fc-YOUR-KEY",
  "url": "https://competitor.com",
  "objective": "Find pricing plans",
  "webhookUrl": "https://hooks.zapier.com/hooks/catch/12345/abcde/",
  "webhookPlatform": "zapier",
  "demoMode": false
}

Who Uses This Actor

• Sales teams — Research prospects before outreach
• Product managers — Monitor competitor features and pricing
• Market researchers — Collect pricing data across industries
• Recruiters — Extract job listings and company info
• Data scientists — Gather training data without writing scrapers

---

Built by John Rippy | Actor Arsenal