Firecrawl Search - AI-Powered Web Search & Scrape

What is Firecrawl?

Firecrawl is a web scraping API that handles JavaScript-heavy websites and anti-bot protection. Instead of managing headless browsers yourself, Firecrawl handles all the complexity.

Why use Firecrawl instead of a regular scraper?

• Handles JavaScript-rendered content automatically
• Works on sites with anti-bot protection
• Returns clean, structured data
• No browser infrastructure to manage

How to Get Your Firecrawl API Key

1. Go to firecrawl.dev (or firecrawl.link/john-rippy for 10% off)
2. Sign up for an account (free tier: 500 credits/month)
3. Go to your dashboard → API Keys
4. Copy your API key
5. Paste it in the firecrawlApiKey field

---

Search the web and get clean, LLM-ready content in one API call. Powered by Firecrawl's /v1/search endpoint. Returns markdown, HTML, or extracted data. Perfect for SEO research, competitor analysis, and AI training data collection. Pay-per-event pricing. Built by John Rippy (https://www.linkedin.com/in/johnrippy/ | https://johnrippy.link/).

Features

• One-Call Search + Scrape: Search Google and get scraped content in a single request
• LLM-Ready Output: Clean markdown format perfect for AI/LLM applications
• Flexible Output: Get markdown, HTML, or both
• Localized Results: Filter by language and country
• Main Content Extraction: Automatically removes ads, navigation, and boilerplate
• Webhook Integration: Send results to Zapier, Make, n8n, or custom endpoints
• Pay-Per-Event Pricing: Only pay for what you use

Quick Start

{
  "query": "best SEO tools 2024",
  "limit": 10,
  "lang": "en",
  "country": "us"
}

Demo Mode

Set demoMode: true to test with sample data (no charges, no API key required). When you're ready for real results, set demoMode: false and provide your Firecrawl API key.

{
  "demoMode": true,
  "query": "SEO best practices"
}

Input Parameters

Parameter	Type	Required	Default	Description
query	string	Yes*	-	Search query to find web pages
limit	integer	No	10	Max results (1-20)
lang	string	No	"en"	Language code (en, es, fr, de, etc.)
country	string	No	"us"	Country code (us, uk, de, fr, etc.)
outputFormat	string	No	"markdown"	Output format: markdown, html, or both
onlyMainContent	boolean	No	true	Extract only main content
firecrawlApiKey	string	Yes**	-	Your Firecrawl API key
demoMode	boolean	No	true	Run with sample data
webhookUrl	string	No	-	Webhook endpoint for results
webhookPlatform	string	No	"custom"	Webhook platform type

* Required when not in demo mode ** Required when not in demo mode. Get a free API key at https://firecrawl.dev

Output Format

All searches return a consistent output structure:

{
  success: boolean;        // Whether the search succeeded
  query: string;           // The search query used
  resultsCount: number;    // Number of results returned
  results: Array<{
    url: string;           // Page URL
    title: string;         // Page title
    description?: string;  // Meta description or snippet
    markdown?: string;     // Clean markdown content
    html?: string;         // Full HTML content
    links?: string[];      // Links found on the page
  }>;
  searchedAt: string;      // ISO timestamp
  error?: string;          // Error message if failed
}

Use Cases

1. SEO Content Research

Find top-ranking content for any keyword and analyze what's working:

{
  "query": "how to improve website speed",
  "limit": 10,
  "outputFormat": "markdown"
}

2. Competitor Analysis

Research competitor content and strategies:

{
  "query": "site:competitor.com pricing",
  "limit": 5
}

3. AI Training Data

Collect clean content for LLM fine-tuning:

{
  "query": "machine learning tutorials",
  "limit": 20,
  "onlyMainContent": true
}

4. Market Research

Gather industry information and trends:

{
  "query": "SaaS pricing trends 2024",
  "limit": 15,
  "country": "us"
}

Webhook Integration

Send results automatically to your automation platform:

{
  "query": "SEO updates",
  "webhookUrl": "https://hooks.zapier.com/hooks/catch/...",
  "webhookPlatform": "zapier"
}

Supported platforms:

• Zapier: Flat payload with _timestamp and _source fields
• Make (Integromat): Nested data and meta structure
• n8n: json wrapper with headers metadata
• Custom: Raw output as-is

---

Pricing

This actor uses pay-per-event billing:

• search_completed: $0.02 per search query

Demo mode runs are free (no API key required).

API Key

Get 10% off at firecrawl.link/john-rippy - Sign up for free tier (500 credits/month) or paid plans.

Firecrawl offers:

• Free tier with 500 credits/month
• Pay-as-you-go pricing
• Enterprise plans for high volume

---

Common Problems & Solutions

"Invalid API key" error

Cause: Your API key is wrong, expired, or doesn't have the right permissions. Fix: Double-check your API key. Make sure you copied it exactly without extra spaces.

"Rate limit exceeded" error

Cause: You've hit the API's rate limits. Fix: Wait a few minutes, then try again. Consider reducing the number of concurrent requests.

Empty or incomplete results

Cause: The target may have anti-scraping protection or the data doesn't exist. Fix:

• Check if the URL/search query is correct
• Try with different parameters
• Some sites may block automated access

Demo data showing instead of real results

Cause: demoMode is still set to true. Fix: Set demoMode: false and provide your API key(s).

---

Built by John Rippy | Actor Arsenal