Firecrawl Agent - AI Autonomous Web Data Extraction

AI-powered autonomous web agent that navigates websites, handles complex interactions, and extracts structured data using natural language goals. Powered by Firecrawl's Agent API. Built by John Rippy (https://www.linkedin.com/in/johnrippy/ | https://johnrippy.link/).

Features

• Automated data collection
• Structured output format
• Error handling
• Pay-per-event billing

Quick Start

{
  "input": "your input here"
}

Demo Mode

Set demoMode: true to test with sample data (no charges). When you're ready for real results, set demoMode: false or omit it.

{
  "demoMode": true,
  ...
}

Input Parameters

Parameter	Type	Required	Default	Description
url	string	Yes*	-	Starting URL for extraction
objective	string	Yes*	-	Natural language description of what to extract
schema	object	No	-	JSON schema for structured output
maxSteps	integer	No	10	Maximum navigation actions
allowBacktracking	boolean	No	true	Allow agent to try alternative paths
firecrawlApiKey	string	Yes*	-	Your Firecrawl API key
demoMode	boolean	No	false	Test with sample data

*Not required in demo mode

---

Output Format

{
  "success": true,
  "url": "https://example.com",
  "objective": "Find pricing plans",
  "data": {
    "plans": [
      {
        "name": "Starter",
        "price": 29,
        "features": ["5 users", "10GB storage"]
      },
      {
        "name": "Pro",
        "price": 99,
        "features": ["Unlimited users", "100GB storage", "Priority support"]
      }
    ]
  },
  "stepsUsed": 4,
  "pagesVisited": [
    "https://example.com",
    "https://example.com/pricing"
  ],
  "extractedAt": "2024-12-23T10:30:00Z"
}

---

Pricing

This actor uses pay-per-event billing:

BYOK (Bring Your Own Key)

Provider	Cost	Purpose
Firecrawl	~$0.01-0.05/extraction	Agent API usage
Apify	~$0.10-0.25/run	Compute

Cost Per Extraction

Complexity	Estimated Cost
Single page	~$0.01-0.02
2-5 pages	~$0.02-0.03
Complex multi-page	~$0.03-0.05

Cost Comparison

Task	Firecrawl Agent	Custom Scraper Development
Extract pricing from 1 site	~$0.03	2-4 hours dev time
Scrape 100 product listings	~$0.50	1-2 days dev time
Monthly extraction (100 sites)	~$5	Maintenance + hosting

No development time. No maintenance. Pay per use.

---

Use Cases

1. Competitive Pricing Research

{
  "url": "https://competitor.com",
  "objective": "Find all pricing plans, their monthly and annual costs, and included features"
}

Returns:

{
  "plans": [
    {
      "name": "Starter",
      "monthlyPrice": 29,
      "annualPrice": 290,
      "features": ["5 users", "10GB storage", "Email support"]
    },
    {
      "name": "Pro",
      "monthlyPrice": 99,
      "annualPrice": 990,
      "features": ["Unlimited users", "100GB storage", "Priority support"]
    }
  ]
}

2. Job Listings Extraction

{
  "url": "https://company.com/careers",
  "objective": "Extract all software engineering job openings with title, location, and requirements"
}

3. Product Catalog Scraping

{
  "url": "https://store.com/products",
  "objective": "Get all products with name, price, description, and availability",
  "schema": {
    "type": "array",
    "items": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "price": { "type": "number" },
        "inStock": { "type": "boolean" }
      }
    }
  }
}

4. Company Research

{
  "url": "https://company.com",
  "objective": "Find the leadership team, their titles, and LinkedIn profiles"
}

5. Event/Conference Extraction

{
  "url": "https://conference.com",
  "objective": "Extract the conference schedule with session titles, speakers, times, and room locations"
}

6. Documentation Scraping

{
  "url": "https://docs.example.com",
  "objective": "Extract all API endpoints with their methods, parameters, and descriptions"
}

---

---

Built by John Rippy | Actor Arsenal