Unified Serper.dev ETL Processor

Discover. Enrich. Outreach. Repeat.

---

Overview

The Unified Serper.dev ETL Processor is a comprehensive business discovery and enrichment tool designed for:

• Lead Generation - Find businesses in specific industries across multiple cities
• Market Research - Analyze business density, ratings, and distribution in target markets
• Competitor Analysis - Discover competitors and their online presence
• Sales Prospecting - Build targeted lists with owner/decision-maker information
• Local SEO Research - Understand local business landscapes

Key Value Proposition

Unlike simple scrapers, this Actor provides a complete ETL pipeline that:

1. Geocodes cities using OpenStreetMap's Nominatim API (free, no API key required)
2. Generates geo-grids to ensure comprehensive coverage of large metropolitan areas
3. Sweeps the area using Serper.dev Maps API to discover businesses
4. Enriches results with owner names and LinkedIn profiles via Serper.dev Search API
5. Deduplicates results using stable business identifiers

---

Features

Core Capabilities

Feature	Description
Multi-City Search	Process multiple cities in a single run
Multi-Keyword Search	Search for multiple business types simultaneously
Geo-Grid Coverage	Systematic grid-based search ensures no businesses are missed
Serper Maps Integration	Discover businesses with full Google Maps data
Serper Search Enrichment	Find owners, founders, and LinkedIn profiles
Smart Deduplication	Prevents duplicate businesses using placeId/cid/coordinates
Configurable Filters	Filter by rating, review count, and website availability
Flexible Limits	Control per-city and total business collection targets

Data Enrichment

• LinkedIn Profile Discovery - Find personal LinkedIn profiles of owners/founders
• LinkedIn Company Pages - Discover company LinkedIn pages
• Owner/Founder Names - Extract owner information from business websites
• Social Media Detection - Identify Facebook, Instagram, and TikTok presence

---

How It Works (ETL Pipeline)

┌──────────────────────────────────────────────────────────────────────────────┐
│                           ETL PIPELINE FLOW                                  │
└──────────────────────────────────────────────────────────────────────────────┘

    INPUT                    EXTRACT                 TRANSFORM              LOAD
  ┌────────┐              ┌──────────┐             ┌──────────┐         ┌────────┐
  │Keywords│              │Nominatim │             │Normalize │         │Dataset │
  │Cities  │──────────────│Geocoding │─────────────│& Filter  │─────────│Output  │
  │Config  │              │          │             │          │         │        │
  └────────┘              └──────────┘             └──────────┘         └────────┘
                               │                        │
                               ▼                        │
                          ┌──────────┐                  │
                          │Geo Grid  │                  │
                          │Generator │                  │
                          └──────────┘                  │
                               │                        │
                               ▼                        │
                          ┌──────────┐                  │
                          │Serper    │                  │
                          │Maps API  │──────────────────┤
                          └──────────┘                  │
                               │                        │
                               ▼                        │
                          ┌──────────┐                  │
                          │Serper    │                  │
                          │Search API│──────────────────┘
                          │(Enrich)  │
                          └──────────┘

---

Input Configuration

Required Parameters

Parameter	Type	Description
serperApiKey	string	Your API key from serper.dev
keywords	array	Business types to search for (e.g., "restaurant", "dentist")
cities	array	Cities to search in (e.g., "Berlin, Germany", "Munich, Germany")

Localization Settings

Parameter	Type	Default	Description
gl	string	"de"	Google country code for localized results
hl	string	"de"	Language for search results

Supported Countries: Argentina, Austria, Australia, Belgium, Brazil, Canada, Switzerland, Chile, China, Colombia, Czech Republic, Germany, Denmark, Egypt, Spain, Finland, France, United Kingdom, Greece, Hong Kong, Hungary, Indonesia, Ireland, Israel, India, Italy, Japan, Kenya, South Korea, Mexico, Malaysia, Nigeria, Netherlands, Norway, New Zealand, Peru, Philippines, Poland, Portugal, Romania, Russia, Saudi Arabia, Sweden, Singapore, Thailand, Turkey, Taiwan, Ukraine, United States, Vietnam, South Africa

Geographic Settings

Parameter	Type	Default	Range	Description
radiusKm	number	20	1-100	Distance from city center to search
stepKm	number	5	0.5-20	Size of grid cells (smaller = more precise but slower)

Collection Targets

Parameter	Type	Default	Range	Description
perCityTarget	integer	300	10-10000	Target businesses per city
maxBusinessesTotal	integer	1200	10-10000	Maximum total businesses across all cities

Business Filters

Parameter	Type	Default	Description
minRating	number	0	Minimum rating (0-5, 0 = no filter)
minReviews	integer	0	Minimum review count (0 = no filter)
mustHaveWebsite	boolean	true	Only include businesses with a website

API Feature Toggles

Parameter	Type	Default	Description
enableMaps	boolean	true	Use Serper Maps API to discover businesses
enableSearchEnrichment	boolean	true	Use Serper Search API for enrichment

Search Enrichment Settings

Parameter	Type	Default	Range	Description
searchNum	integer	5	1-100	Results per search query (>10 uses 2 credits)
maxSearchQueriesPerBusiness	integer	10	1-20	Max searches per business
maxSearchQueriesTotal	integer	800	1-1000000	Total search limit across all businesses
searchTemplates	array	(see below)	-	Custom search query templates

Developer Mode

Parameter	Type	Default	Description
rawSearchResults	boolean	false	Returns raw search results without processing. Ideal for LLM analysis.

🔬 Developer Mode (rawSearchResults: true):

• Returns Maps data + raw search results from Serper
• Disables website scraping (no socials extraction)
• Disables enrichment logic (no owner/LinkedIn matching)
• Keeps all search template queries and their results
• Perfect for feeding data to LLMs for custom analysis

Example Input JSON

{
  "serperApiKey": "your-api-key-here",
  "keywords": ["restaurant", "cafe", "bar"],
  "cities": ["Frankfurt, Germany", "Munich, Germany"],
  "gl": "de",
  "hl": "de",
  "radiusKm": 15,
  "stepKm": 4,
  "perCityTarget": 200,
  "maxBusinessesTotal": 500,
  "minRating": 4.0,
  "minReviews": 10,
  "mustHaveWebsite": true,
  "enableMaps": true,
  "enableSearchEnrichment": true,
  "searchNum": 5,
  "maxSearchQueriesPerBusiness": 2,
  "maxSearchQueriesTotal": 400
}

---

Search Templates

Search templates define how the Actor searches for owner/LinkedIn information. They use placeholders that are replaced with business data.

Available Placeholders

Placeholder	Source	Description
{{name}}	title	Business name (e.g., "Café Müller")
{{city}}	Input city	The city you're searching (e.g., "Frankfurt")
{{site}}	website	Full website URL (e.g., "https://cafe-mueller.de")
{{category}}	keyword or type	Search keyword or first business type

Default Templates

The Actor comes with owner/LinkedIn-focused default templates:

site:linkedin.com/in/ "{{name}}" ("Owner" OR "Founder" OR "CEO" OR "Geschäftsführer" OR "Inhaber")
site:linkedin.com/in/ "{{name}}" {{city}}
site:linkedin.com/company/ "{{name}}"
site:{{site}} ("Impressum" OR "Legal Notice" OR "Kontakt" OR "Team" OR "Über uns" OR "About")

Template Behavior

• Website-dependent templates: Templates containing {{site}} are automatically skipped for businesses without a website
• Query validation: Empty or very short queries (< 6 characters) are skipped
• Intent classification: Each template is classified by intent (linkedin_profile, linkedin, facebook, instagram, general)

Custom Templates Example

{
  "searchTemplates": [
    "\"CEO\" \"{{name}}\" site:linkedin.com/in",
    "\"{{name}}\" \"{{city}}\" \"contact\" site:{{site}}",
    "\"{{name}}\" owner email",
    "\"{{name}}\" \"{{city}}\" site:xing.com"
  ]
}

---

Output Data

The Actor produces two output formats depending on the enableSearchEnrichment setting:

Maps-Only Mode (enableSearchEnrichment: false)

Field	Type	Description
title	string	Business name
address	string	Full address
rating	number	Google rating (0-5)
ratingCount	number	Number of reviews
website	string	Business website URL
phoneNumber	string	Phone number from Google Maps
type	string	Primary business category
types	array	All business categories
cid	string	Google CID
fid	string	Google FID
placeId	string	Google Place ID

Enriched Mode (enableSearchEnrichment: true)

All fields from Maps-Only mode, plus:

Field	Type	Description
socials	array	Objects with platform and url (linkedin, facebook, etc.)
ownerName	string	Name of the owner/founder found
ownerTitle	string	Title (CEO, Owner, Founder, etc.)
linkedinProfile	string	LinkedIn profile URL of the owner
companyLinkedIn	string	LinkedIn company page URL
ownerSource	string	Source of owner data (LinkedIn, Website Impressum)
businessId	string	Unique deduplication ID
searchResults	array	Objects with query and results[] - grouped by search template

Example Output (Enriched)

{
  "title": "Tech Solutions GmbH",
  "address": "Alexanderplatz 1, 10178 Berlin, Germany",
  "rating": 4.8,
  "ratingCount": 156,
  "website": "https://tech-solutions.de",
  "phoneNumber": "+49 30 123456",
  "type": "Software Company",
  "types": ["Software Company", "IT Consultant"],
  "cid": "1234567890",
  "placeId": "ChIJv026La8OvUcRs2_LOGWaHA4",
  "socials": [
    { "platform": "linkedin", "url": "https://linkedin.com/company/tech-solutions" },
    { "platform": "facebook", "url": "https://facebook.com/techsolutions" },
    { "platform": "instagram", "url": "https://instagram.com/techsolutions" }
  ],
  "ownerName": "Hans Müller",
  "ownerTitle": "CEO",
  "linkedinProfile": "https://linkedin.com/in/hans-mueller",
  "companyLinkedIn": "https://linkedin.com/company/tech-solutions",
  "ownerSource": "LinkedIn",
  "businessId": "place_ChIJv026La8OvUcRs2_LOGWaHA4",
  "searchResults": [
    {
      "query": "site:linkedin.com/in/ \"Tech Solutions GmbH\" (\"Owner\" OR \"Founder\" OR \"CEO\")",
      "results": [
        {
          "position": 1,
          "title": "Hans Müller - CEO - Tech Solutions GmbH | LinkedIn",
          "link": "https://linkedin.com/in/hans-mueller",
          "snippet": "CEO at Tech Solutions GmbH. Berlin, Germany. 500+ connections."
        },
        {
          "position": 2,
          "title": "Maria Schmidt - CTO - Tech Solutions GmbH | LinkedIn",
          "link": "https://linkedin.com/in/maria-schmidt",
          "snippet": "CTO at Tech Solutions GmbH. Leading engineering team."
        }
      ]
    },
    {
      "query": "site:linkedin.com/company/ \"Tech Solutions GmbH\"",
      "results": [
        {
          "position": 1,
          "title": "Tech Solutions GmbH | LinkedIn",
          "link": "https://linkedin.com/company/tech-solutions",
          "snippet": "Tech Solutions GmbH | 50 followers on LinkedIn. Software development..."
        }
      ]
    }
  ]
}

---

API Costs & Limits

Serper.dev Credits

Get your API key at serper.dev.

🔰	Credits
🎁 New users	2,500 free credits
💳 Top-up	$50 = 50,000 credits

Credit Usage per API Call

API Endpoint	Credits	Returns
Maps API (Google Maps)	3 credits	Up to 20 businesses
Search API (Google Search)	1 credit	Enrichment results

💡 Tip: Maps calls cost 3x more than Search calls, so optimize your grid settings!

What Can You Get with $50?

With 50,000 credits, you can collect approximately:

Mode	Businesses	Search Queries	Total Credits
Maps-Only	~16,000	0	48,000
With Enrichment	~5,000	10,000	50,000

Calculation based on: 3 credits per Maps call (20 results), 1 credit per Search query, 2 queries per business.

Cost Formulas

Maps Credits = Maps Calls × 3 Search Credits = Businesses × Queries per Business

Maps Calls ≈ π × (radiusKm ÷ stepKm)² × Cities × Keywords × Pages

Example: Frankfurt Restaurants

Config: 1 city, 1 keyword, radius 20km, step 10km, 500 businesses with enrichment

Step	Calculation	Credits
Grid cells	π × (20 ÷ 10)² = ~13 cells	-
Maps calls	13 cells × 3 pages = 39 calls	39 × 3 = 117
Search calls	500 businesses × 2 queries	500 × 2 = 1,000
Total Serper		1,117 credits

Apify Platform Costs

Running this Actor on Apify costs approximately:

Results	Apify Cost
1,000 businesses	~$0.10
5,000 businesses	~$0.50
10,000 businesses	~$1.00

⚠️ Based on actual usage tests. Costs may vary depending on enrichment settings and website scraping time.

Rate Limiting

The Actor implements automatic rate limiting with Bottleneck:

Service	Concurrency	Min Delay	Purpose
Serper API	5 concurrent	100ms	Prevent rate limit errors
Nominatim	1 concurrent	1.2 seconds	Respect OpenStreetMap guidelines
Website Scraper	3 concurrent	200ms	Prevent overload

Production Features

Feature	Description
Geocoding Cache	Results cached in KeyValueStore to avoid repeated API calls
Retry Logic	Exponential backoff for 5xx, 429, and network errors
State Persistence	Progress saved every 100 leads for crash recovery
Graceful Shutdown	Clean exit on SIGTERM/SIGINT with state save
API Key Validation	Format verified (32-64 hex characters)

---

Examples

Basic Usage: Single City, Single Keyword

{
  "serperApiKey": "your-api-key",
  "keywords": ["dentist"],
  "cities": ["Berlin, Germany"],
  "gl": "de",
  "hl": "de",
  "perCityTarget": 100,
  "enableSearchEnrichment": false
}

Advanced Usage: Multi-City with Enrichment

{
  "serperApiKey": "your-api-key",
  "keywords": ["software company", "IT consulting", "web agency"],
  "cities": [
    "Frankfurt, Germany",
    "Munich, Germany",
    "Hamburg, Germany",
    "Berlin, Germany"
  ],
  "gl": "de",
  "hl": "de",
  "radiusKm": 25,
  "stepKm": 5,
  "perCityTarget": 150,
  "maxBusinessesTotal": 500,
  "minRating": 4.0,
  "minReviews": 5,
  "mustHaveWebsite": true,
  "enableMaps": true,
  "enableSearchEnrichment": true,
  "searchNum": 5,
  "maxSearchQueriesPerBusiness": 3,
  "maxSearchQueriesTotal": 1000,
  "searchTemplates": [
    "\"CEO\" \"{{name}}\" site:linkedin.com/in",
    "\"CTO\" \"{{name}}\" site:linkedin.com/in",
    "\"{{name}}\" \"{{city}}\" site:linkedin.com/company"
  ]
}

US Market Example

{
  "serperApiKey": "your-api-key",
  "keywords": ["real estate agent", "mortgage broker"],
  "cities": [
    "Los Angeles, CA",
    "San Francisco, CA",
    "San Diego, CA"
  ],
  "gl": "us",
  "hl": "en",
  "radiusKm": 30,
  "stepKm": 6,
  "perCityTarget": 200,
  "maxBusinessesTotal": 500,
  "minRating": 4.5,
  "mustHaveWebsite": true,
  "searchTemplates": [
    "\"owner\" \"{{name}}\" site:{{site}}",
    "\"{{name}}\" \"{{city}}\" (\"Owner\" OR \"Broker\" OR \"Agent\") site:linkedin.com/in"
  ]
}

Developer Mode: Raw Search for LLM Analysis

{
  "serperApiKey": "your-api-key",
  "keywords": ["restaurant"],
  "cities": ["Berlin, Germany"],
  "gl": "de",
  "hl": "de",
  "perCityTarget": 50,
  "enableSearchEnrichment": true,
  "rawSearchResults": true,
  "searchNum": 10,
  "maxSearchQueriesPerBusiness": 3,
  "searchTemplates": [
    "site:linkedin.com/in/ \"{{name}}\" (\"Owner\" OR \"Founder\" OR \"CEO\")",
    "site:linkedin.com/company/ \"{{name}}\"",
    "\"{{name}}\" {{city}} owner founder contact"
  ]
}

💡 This returns raw search results that can be processed by an LLM to extract owner names, roles, and other information with custom logic.

---

Understanding radiusKm and stepKm

These two parameters control how the Actor searches for businesses around each city:

radiusKm = Maximum distance from city center to search (in kilometers) stepKm = Distance between search points in the grid (in kilometers)

How It Works

1. The Actor geocodes the city → gets center coordinates (lat/lon)
2. Creates a grid of search points within the radius
3. Each point = 1 API call to Serper (returns up to 20 results)

Simple Formula

Number of API calls per city ≈ π × (radiusKm ÷ stepKm)²

Configuration Presets

Preset	radiusKm	stepKm	API Calls/City	Best For
Quick	20	20	~5	Fast testing, small towns
Balanced	30	15	~13	Standard city coverage
Thorough	50	25	~13	Large metropolitan areas
Extensive	60	30	~13	Very large coverage area

---

⚠️ Avoid: radiusKm: 60, stepKm: 8 → 177 API calls per city! Too expensive.

The Golden Rule

stepKm should be at least 1/3 to 1/2 of radiusKm

Bad	Good
radius: 60, step: 8 (177 calls)	radius: 60, step: 30 (13 calls)
radius: 30, step: 3 (314 calls)	radius: 30, step: 15 (13 calls)
radius: 20, step: 2 (314 calls)	radius: 20, step: 10 (13 calls)

City Size Recommendations

City Size	Population	radiusKm	stepKm
Small town	< 100k	10-15	10-15
Medium city	100k-500k	15-25	15-20
Large city	500k-2M	25-40	20-25
Metropolitan	> 2M	40-60	25-30

Balancing Coverage vs. API Costs

1. Start small: Test with one city and one keyword first
2. Adjust step size: Larger steps = fewer API calls but may miss businesses
3. Use filters: minRating and minReviews reduce low-quality results
4. Limit enrichment: Set maxSearchQueriesTotal to control search costs

When to Enable/Disable Enrichment

Enable enrichment when:

• You need owner/decision-maker contact information
• LinkedIn profiles are valuable for your use case
• You have sufficient API budget

Disable enrichment when:

• You only need basic business data (name, address, phone, website)
• You're doing initial market research
• You want to minimize API costs

Optimizing Search Templates

1. Use site: operator: Constrains results to specific domains
2. Include location: Adding address improves relevance
3. Use OR operators: Combine multiple terms for broader coverage
4. Test templates: Run small batches to verify template effectiveness

---

Built with 🩶 for the Apify community 🫡