Unified Serper.dev ETL Processor

Overview

The Unified Serper.dev ETL Processor is a comprehensive business discovery and enrichment tool designed for:

• Lead Generation - Find businesses in specific industries across multiple cities
• Market Research - Analyze business density, ratings, and distribution in target markets
• Competitor Analysis - Discover competitors and their online presence
• Sales Prospecting - Build targeted lists with owner/decision-maker information
• Local SEO Research - Understand local business landscapes

Key Value Proposition

Unlike simple scrapers, this Actor provides a complete ETL pipeline that:

1. Geocodes cities using OpenStreetMap's Nominatim API (free, no API key required)
2. Generates geo-grids to ensure comprehensive coverage of large metropolitan areas
3. Sweeps the area using Serper.dev Maps API to discover businesses
4. Enriches results with owner names and LinkedIn profiles via Serper.dev Search API
5. Deduplicates results using stable business identifiers

---

Features

Core Capabilities

Feature	Description
Multi-City Search	Process multiple cities in a single run
Multi-Keyword Search	Search for multiple business types simultaneously
Geo-Grid Coverage	Systematic grid-based search ensures no businesses are missed
Serper Maps Integration	Discover businesses with full Google Maps data
Serper Search Enrichment	Find owners, founders, and LinkedIn profiles
Smart Deduplication	Prevents duplicate businesses using placeId/cid/coordinates
Configurable Filters	Filter by rating, review count, and website availability
Flexible Limits	Control per-city and total business collection targets

Data Enrichment

• LinkedIn Profile Discovery - Find personal LinkedIn profiles of owners/founders
• LinkedIn Company Pages - Discover company LinkedIn pages
• Owner/Founder Names - Extract owner information from business websites
• Social Media Detection - Identify Facebook and Instagram presence

---

How It Works (ETL Pipeline)

┌─────────────────────────────────────────────────────────────────────────────┐
│                           ETL PIPELINE FLOW                                 │
└─────────────────────────────────────────────────────────────────────────────┘

    INPUT                    EXTRACT                 TRANSFORM              LOAD
  ┌────────┐              ┌──────────┐             ┌──────────┐         ┌────────┐
  │Keywords│              │Nominatim │             │Normalize │         │Dataset │
  │Cities  │──────────────│Geocoding │─────────────│& Filter  │─────────│Output  │
  │Config  │              │          │             │          │         │        │
  └────────┘              └──────────┘             └──────────┘         └────────┘
                               │                        │
                               ▼                        │
                          ┌──────────┐                  │
                          │Geo Grid  │                  │
                          │Generator │                  │
                          └──────────┘                  │
                               │                        │
                               ▼                        │
                          ┌──────────┐                  │
                          │Serper    │                  │
                          │Maps API  │──────────────────┤
                          └──────────┘                  │
                               │                        │
                               ▼                        │
                          ┌──────────┐                  │
                          │Serper    │                  │
                          │Search API│──────────────────┘
                          │(Enrich)  │
                          └──────────┘

Step-by-Step Process

Step 1: Nominatim Geocoding

• Each city name is converted to geographic coordinates (latitude/longitude)
• Uses OpenStreetMap's free Nominatim API
• Rate-limited to 1 request per 1.1 seconds to respect API guidelines

Step 2: Geo Grid Generation

• Creates a grid of search points around the city center
• Grid size determined by radiusKm (distance from center) and stepKm (cell size)
• Only cells within the circular radius are included
• Example: 20km radius with 5km step = ~49 grid cells

Step 3: Serper Maps Sweep

• For each grid cell and keyword combination, queries Serper Maps API
• Collects business data: name, address, rating, website, phone, coordinates
• Applies filters (rating, reviews, website requirement)
• Deduplicates using stable business IDs

Step 4: Serper Search Enrichment (Optional)

• For each collected business, runs customizable search queries
• Default templates focus on finding owners and LinkedIn profiles
• Results include social media detection flags

---

Input Configuration

Required Parameters

Parameter	Type	Description
serperApiKey	string	Your API key from serper.dev
keywords	array	Business types to search for (e.g., "restaurant", "dentist")
cities	array	Cities to search in (e.g., "Berlin, Germany", "Munich, Germany")

Localization Settings

Parameter	Type	Default	Description
gl	string	"de"	Google country code for localized results
hl	string	"de"	Language for search results

Supported Countries: Argentina, Austria, Australia, Belgium, Brazil, Canada, Switzerland, Chile, China, Colombia, Czech Republic, Germany, Denmark, Egypt, Spain, Finland, France, United Kingdom, Greece, Hong Kong, Hungary, Indonesia, Ireland, Israel, India, Italy, Japan, Kenya, South Korea, Mexico, Malaysia, Nigeria, Netherlands, Norway, New Zealand, Peru, Philippines, Poland, Portugal, Romania, Russia, Saudi Arabia, Sweden, Singapore, Thailand, Turkey, Taiwan, Ukraine, United States, Vietnam, South Africa

Geographic Settings

Parameter	Type	Default	Range	Description
radiusKm	number	20	1-100	Distance from city center to search
stepKm	number	5	0.5-20	Size of grid cells (smaller = more precise but slower)

Collection Targets

Parameter	Type	Default	Range	Description
perCityTarget	integer	300	10-1000	Target businesses per city
maxBusinessesTotal	integer	1200	10-10000	Maximum total businesses across all cities

Business Filters

Parameter	Type	Default	Description
minRating	number	0	Minimum rating (0-5, 0 = no filter)
minReviews	integer	0	Minimum review count (0 = no filter)
mustHaveWebsite	boolean	true	Only include businesses with a website

API Feature Toggles

Parameter	Type	Default	Description
enableMaps	boolean	true	Use Serper Maps API to discover businesses
enableSearchEnrichment	boolean	true	Use Serper Search API for enrichment

Search Enrichment Settings

Parameter	Type	Default	Range	Description
searchNum	integer	5	1-20	Results per search query
maxSearchQueriesPerBusiness	integer	2	1-10	Max searches per business
maxSearchQueriesTotal	integer	800	1-5000	Total search limit across all businesses
searchTemplates	array	(see below)	-	Custom search query templates

Advanced Options

Parameter	Type	Default	Description
includeRaw	boolean	false	Include raw API responses in output (for debugging)

Example Input JSON

{
  "serperApiKey": "your-api-key-here",
  "keywords": ["restaurant", "cafe", "bar"],
  "cities": ["Frankfurt, Germany", "Munich, Germany"],
  "gl": "de",
  "hl": "de",
  "radiusKm": 15,
  "stepKm": 4,
  "perCityTarget": 200,
  "maxBusinessesTotal": 500,
  "minRating": 4.0,
  "minReviews": 10,
  "mustHaveWebsite": true,
  "enableMaps": true,
  "enableSearchEnrichment": true,
  "searchNum": 5,
  "maxSearchQueriesPerBusiness": 2,
  "maxSearchQueriesTotal": 400,
  "includeRaw": false
}

---

Search Templates

Search templates define how the Actor searches for owner/LinkedIn information. They use placeholders that are replaced with business data.

Available Placeholders

Placeholder	Description	Example
{{name}}	Business name	"Café Müller"
{{city}}	Business city	"Frankfurt"
{{domainRoot}}	Root domain from website	"cafe-mueller.de"
{{category}}	Business type/category	"Restaurant"
{{site}}	Same as domainRoot	"cafe-mueller.de"

Default Templates

The Actor comes with owner/LinkedIn-focused default templates:

"owner" "{{name}}" site:{{site}}
"Inhaber" "{{name}}" site:{{site}}
"{{name}}" "{{city}}" ("Owner" OR "Founder" OR "Geschäftsführer" OR "Inhaber") site:linkedin.com/in
"{{name}}" "{{city}}" site:linkedin.com/company

Template Behavior

• Website-dependent templates: Templates containing {{site}} or {{domainRoot}} are automatically skipped for businesses without a website
• Query validation: Empty or very short queries (< 6 characters) are skipped
• Intent classification: Each template is classified by intent (linkedin_profile, linkedin, facebook, instagram, general)

Custom Templates Example

{
  "searchTemplates": [
    "\"CEO\" \"{{name}}\" site:linkedin.com/in",
    "\"{{name}}\" \"{{city}}\" \"contact\" site:{{site}}",
    "\"{{name}}\" owner email",
    "\"{{name}}\" \"{{city}}\" site:xing.com"
  ]
}

---

Output Data

The Actor produces two types of records in the dataset:

Maps Records (Business Data)

Records with _source: "maps" contain business information:

Field	Type	Description
_source	string	Always "maps"
_city	string	Input city name
_keyword	string	Search keyword used
_serperType	string	Always "maps"
_cellIndex	integer	Geo grid cell index
_timestamp	string	ISO 8601 timestamp
title	string	Business name
address	string	Full address
city	string	Extracted city from address
rating	number	Google rating (1-5)
ratingCount	number	Number of reviews
priceLevel	string	Price indicator ($, $$, etc.)
types	array	Business type categories
description	string	Business description
openingHours	string/object	Opening hours
website	string	Business website URL
phoneNumber	string	Phone number
type	string	Primary business category
thumbnailUrl	string	Business image URL
latitude	number	Geographic latitude
longitude	number	Geographic longitude
cid	string	Google Maps CID
fid	string	Google Maps FID
placeId	string	Google Place ID
businessId	string	Unique identifier (generated)
domainRoot	string	Root domain from website
country	string	Country code

Search Records (Enrichment Data)

Records with _source: "search" contain enrichment results:

Field	Type	Description
_source	string	Always "search"
_city	string	Input city name
_keyword	string	Original search keyword
_serperType	string	Always "search"
_businessId	string	Reference to Maps business
_businessName	string	Linked business name
_businessWebsite	string	Linked business website
_query	string	Full search query used
_intent	string	Query intent classification
_position	number	Position in search results
_timestamp	string	ISO 8601 timestamp
title	string	Search result title
url	string	Result URL
domain	string	Result domain
snippet	string	Result snippet/description
favicon	string	Favicon URL
date	string	Result date (if available)
isLinkedIn	boolean	Result is from LinkedIn
isLinkedInProfile	boolean	Result is a LinkedIn profile
isFacebook	boolean	Result is from Facebook
isInstagram	boolean	Result is from Instagram

Example Output

Maps Record

{
  "_source": "maps",
  "_city": "Frankfurt, Germany",
  "_keyword": "restaurant",
  "_serperType": "maps",
  "_cellIndex": 12,
  "_timestamp": "2024-01-15T10:30:00.000Z",
  "title": "Restaurant Zum Goldenen Löwen",
  "address": "Hauptstraße 42, 60311 Frankfurt am Main, Germany",
  "city": "Frankfurt am Main",
  "rating": 4.5,
  "ratingCount": 234,
  "priceLevel": "$$",
  "website": "https://www.goldener-loewe-ffm.de",
  "phoneNumber": "+49 69 1234567",
  "type": "Restaurant",
  "latitude": 50.1109,
  "longitude": 8.6821,
  "placeId": "ChIJxxxxxxxxxx",
  "businessId": "place_ChIJxxxxxxxxxx",
  "domainRoot": "goldener-loewe-ffm.de",
  "country": "DE"
}

Search Record

{
  "_source": "search",
  "_city": "Frankfurt, Germany",
  "_keyword": "restaurant",
  "_serperType": "search",
  "_businessId": "place_ChIJxxxxxxxxxx",
  "_businessName": "Restaurant Zum Goldenen Löwen",
  "_businessWebsite": "https://www.goldener-loewe-ffm.de",
  "_query": "\"Restaurant Zum Goldenen Löwen\" \"Frankfurt\" site:linkedin.com/in",
  "_intent": "linkedin_profile",
  "_position": 1,
  "_timestamp": "2024-01-15T10:35:00.000Z",
  "title": "Hans Müller - Owner at Restaurant Zum Goldenen Löwen | LinkedIn",
  "url": "https://www.linkedin.com/in/hans-mueller-12345",
  "domain": "linkedin.com",
  "snippet": "View Hans Müller's profile on LinkedIn, the world's largest professional community...",
  "isLinkedIn": true,
  "isLinkedInProfile": true,
  "isFacebook": false,
  "isInstagram": false
}

---

Run Metrics

At the end of each run, the Actor outputs a comprehensive summary:

═══════════════════════════════════════════════════════════════
  🦑Unified Serper.dev ETL v8.8🦑
═══════════════════════════════════════════════════════════════
CONFIG
  • Keywords            │ 3
  • Cities              │ 2
  • Radius / Step       │ 15km / 4km
  • gl / hl             │ de / de
  • Maps / Search / Raw │ ON / ON / OFF
GEO
  • Geocoded cities     │ 2 / 2
  • Grid cells          │ 82
MAPS
  • Cities processed    │ 2 / 2
  • Businesses          │ 487
  • API calls           │ 246
SEARCH
  • Records             │ 1,842
  • API calls           │ 974
RUN
  • Duration            │ 12m 34s
  • Raw payloads        │ EXCLUDED
───────────────────────────────────────────────────────────────

---

API Costs & Limits

Serper.dev Pricing

Serper.dev charges per API call. As of the latest pricing:

• Maps API: ~$0.001 per call
• Search API: ~$0.001 per call

Check serper.dev/pricing for current rates.

Cost Estimation Formula

Maps API Calls ≈ (Grid Cells per City) × (Number of Cities) × (Number of Keywords)
Search API Calls ≈ (Businesses Collected) × (Queries per Business)

Grid Cells ≈ π × (radiusKm / stepKm)²

Example Calculation:

• 2 cities, 3 keywords, 15km radius, 4km step
• Grid cells per city ≈ 3.14 × (15/4)² ≈ 44 cells
• Maps calls ≈ 44 × 2 × 3 = 264 calls
• If 400 businesses collected with 2 queries each = 800 search calls
• Total: ~1,064 API calls ≈ $1.06

Rate Limiting

The Actor implements automatic rate limiting:

• Nominatim: 1.1 second delay between calls
• Serper APIs: 250ms delay between calls
• Error recovery: 500ms delay after failed requests

---

Examples

Basic Usage: Single City, Single Keyword

{
  "serperApiKey": "your-api-key",
  "keywords": ["dentist"],
  "cities": ["Berlin, Germany"],
  "gl": "de",
  "hl": "de",
  "perCityTarget": 100,
  "enableSearchEnrichment": false
}

Advanced Usage: Multi-City with Enrichment

{
  "serperApiKey": "your-api-key",
  "keywords": ["software company", "IT consulting", "web agency"],
  "cities": [
    "Frankfurt, Germany",
    "Munich, Germany",
    "Hamburg, Germany",
    "Berlin, Germany"
  ],
  "gl": "de",
  "hl": "de",
  "radiusKm": 25,
  "stepKm": 5,
  "perCityTarget": 150,
  "maxBusinessesTotal": 500,
  "minRating": 4.0,
  "minReviews": 5,
  "mustHaveWebsite": true,
  "enableMaps": true,
  "enableSearchEnrichment": true,
  "searchNum": 5,
  "maxSearchQueriesPerBusiness": 3,
  "maxSearchQueriesTotal": 1000,
  "searchTemplates": [
    "\"CEO\" \"{{name}}\" site:linkedin.com/in",
    "\"CTO\" \"{{name}}\" site:linkedin.com/in",
    "\"{{name}}\" \"{{city}}\" site:linkedin.com/company"
  ]
}

US Market Example

{
  "serperApiKey": "your-api-key",
  "keywords": ["real estate agent", "mortgage broker"],
  "cities": [
    "Los Angeles, CA",
    "San Francisco, CA",
    "San Diego, CA"
  ],
  "gl": "us",
  "hl": "en",
  "radiusKm": 30,
  "stepKm": 6,
  "perCityTarget": 200,
  "maxBusinessesTotal": 500,
  "minRating": 4.5,
  "mustHaveWebsite": true,
  "searchTemplates": [
    "\"owner\" \"{{name}}\" site:{{site}}",
    "\"{{name}}\" \"{{city}}\" (\"Owner\" OR \"Broker\" OR \"Agent\") site:linkedin.com/in"
  ]
}

---

Tips & Best Practices

Choosing Radius and Step Size

City Size	Recommended Radius	Recommended Step
Small town (<100k pop)	5-10 km	2-3 km
Medium city (100k-500k)	10-20 km	4-5 km
Large city (500k-1M)	15-25 km	5-6 km
Metropolitan area (>1M)	20-40 km	6-8 km

Balancing Coverage vs. API Costs

1. Start small: Test with one city and one keyword first
2. Adjust step size: Larger steps = fewer API calls but may miss businesses
3. Use filters: minRating and minReviews reduce low-quality results
4. Limit enrichment: Set maxSearchQueriesTotal to control search costs

When to Enable/Disable Enrichment

Enable enrichment when:

• You need owner/decision-maker contact information
• LinkedIn profiles are valuable for your use case
• You have sufficient API budget

Disable enrichment when:

• You only need basic business data (name, address, phone, website)
• You're doing initial market research
• You want to minimize API costs

Optimizing Search Templates

1. Use site: operator: Constrains results to specific domains
2. Include location: Adding {{city}} improves relevance
3. Use OR operators: Combine multiple terms for broader coverage
4. Test templates: Run small batches to verify template effectiveness

---

Built with ❤️ for the Apify community 🫡