# Google Search SERP Scraper $8/1k (`autoharvestor/google-search-serp-scraper`) Actor

Scrape Google Search Engine Results Pages (SERPs). 🔥 $8/1k results 🚀 Select the country, language or precise location and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API for SEO research, rank tracking, and content gap workflows.

- **URL**: https://apify.com/autoharvestor/google-search-serp-scraper.md
- **Developed by:** [Dipendra KC](https://apify.com/autoharvestor) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $8.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google Search SERP Scraper

A production-grade, ultra-fast Apify Actor that scrapes Google Search Engine Results Pages (SERPs) and returns structured data. Functions as a reliable **Google SERP API** for SEO rank tracking, market research, content-gap analysis, and competitive intelligence.

***

### ✨ Features

- ⚡ **High-Speed Static Scraping**: Built on Crawlee's `CheerioCrawler` (pure HTTP fetching with zero browser overhead) tuned for high concurrency.
- 🛡️ **Hard AI Overview Exclusion**: Strictly purges and excludes Google AI Overviews, AI Mode, and generative answer elements from extracted data.
- 🌐 **Apify Google SERP Proxy Built-in**: Utilizes Apify's dedicated `GOOGLE_SERP` proxy group (`groups-GOOGLE_SERP`), routing requests to the appropriate regional domain (`google.co.uk`, `google.de`, `google.fr`, etc.) with fallback to `RESIDENTIAL`.
- 📍 **Precision Geo-Targeting (UULE)**: Converts free-text location names (e.g. `Austin,Texas,United States`) into Google's canonical `uule` parameter.
- 📦 **Comprehensive SERP Extraction**:
  - **Organic Results**: Position, title, destination URL, displayed URL / breadcrumbs, description/snippet, sitelinks, and rich snippets (ratings, review counts, prices, dates).
  - **Paid Ads**: Placement (`top` vs `bottom`), position, advertiser domain, title, ad URL, description, callout extensions, and ad sitelinks.
  - **People Also Ask (PAA)**: Question text, answer preview, and source citation URL.
  - **Related Searches**: Suggested search queries.
  - **Knowledge Graph / Answer Box**: Title, subtitle, description, source URL, key-value attributes, and entity images.
  - **Google Shopping**: Product title, price, currency, merchant/seller, rating, and product URL.
- 📊 **Throughput Tracking & Summaries**: Emits live queries/sec and pages/sec metrics, with run summaries stored in the Key-Value Store (`RESULTS-SUMMARY` and `OUTPUT`).

***

### 📥 Input Parameters

| Field | Type | Default | Description |
|---|---|---|---|
| `queries` | Array of strings | `["best running shoes"]` | Search keywords to run (accepts array or one query per line). |
| `resultsPerPage` | Integer (`10, 20, 30, 40, 50, 100`) | `10` | Number of results requested per page (`num` parameter). |
| `maxPagesPerQuery` | Integer | `1` | Pagination depth per query. |
| `countryCode` | String (ISO 3166-1 alpha-2) | `"us"` | Regional country code. Determines regional Google domain (`google.de`, `google.co.uk`, etc.) and `gl` param. |
| `languageCode` | String (ISO 639-1) | `"en"` | Search interface language (`hl` parameter). |
| `location` | String | `""` | Canonical location string (e.g. `Berlin,Germany`). Encoded into Google `uule`. |
| `device` | Enum (`desktop`, `mobile`) | `"desktop"` | Emulated client device and User-Agent headers. |
| `includeOrganicResults` | Boolean | `true` | Extract organic search results. |
| `includePaidAds` | Boolean | `true` | Extract sponsored top and bottom ads. |
| `includePeopleAlsoAsk` | Boolean | `true` | Extract People Also Ask Q\&As. |
| `includeRelatedSearches` | Boolean | `true` | Extract related search queries. |
| `includeKnowledgeGraph` | Boolean | `true` | Extract Knowledge Graph and answer box details. |
| `includeShoppingResults` | Boolean | `true` | Extract Google Shopping product cards. |
| `includeSiteLinks` | Boolean | `true` | Extract sitelinks under organic results and ads. |
| `excludeAiOverview` | Boolean | `true` (Locked) | Hard exclusion. Discards all Google AI Overview / AI Mode blocks. |
| `maxConcurrency` | Integer | `30` | Maximum parallel requests (auto-scaled). |
| `proxyConfiguration` | Object | `{ useApifyProxy: true, apifyProxyGroups: ["GOOGLE_SERP"] }` | Apify proxy group settings. |
| `outputFormat` | Enum (`dataset`, `dataset+csv`, `dataset+json`) | `"dataset"` | Storage format for results. |
| `debugMode` | Boolean | `false` | Save raw SERP HTML snapshots to Key-Value Store under `DEBUG-*`. |

***

### 📤 Output Structure

Data is streamed into the default **Apify Dataset** as each SERP page finishes loading.

#### Example Organic Result Item

```json
{
  "query": "best running shoes",
  "page": 1,
  "resultType": "organic",
  "position": 1,
  "title": "The 10 Best Running Shoes of 2025, Tested and Reviewed",
  "url": "https://www.runnersworld.com/gear/a20865904/best-running-shoes/",
  "displayedUrl": "https://www.runnersworld.com > gear",
  "description": "We tested dozens of road and trail shoes to find the best models for every type of runner, budget, and foot strike.",
  "sitelinks": [
    {
      "title": "Best Cushioned Shoes",
      "url": "https://www.runnersworld.com/gear/cushioned-shoes"
    }
  ],
  "richSnippet": {
    "rating": 4.8,
    "reviewCount": 124,
    "date": "Jan 12, 2025"
  },
  "searchParameters": {
    "countryCode": "us",
    "languageCode": "en",
    "device": "desktop",
    "googleDomain": "google.com"
  },
  "timestamp": "2026-09-17T12:00:00.000Z"
}
```

#### Example Paid Ad Item

```json
{
  "query": "running shoes",
  "page": 1,
  "resultType": "ad",
  "position": 1,
  "title": "Shop Top Running Shoes | Official Nike Store",
  "url": "https://www.nike.com/running",
  "displayedUrl": "https://www.nike.com",
  "description": "Experience comfort and speed. Browse the newest road racing and trail running shoes with free shipping.",
  "adInfo": {
    "placement": "top",
    "advertiserDomain": "nike.com",
    "callouts": ["Free Shipping on $50+", "60-Day Returns"]
  },
  "searchParameters": {
    "countryCode": "us",
    "languageCode": "en",
    "device": "desktop",
    "googleDomain": "google.com"
  },
  "timestamp": "2026-09-17T12:00:00.000Z"
}
```

***

### 🚀 How to Run via API

#### 1. Apify API (cURL)

```bash
curl --request POST 'https://api.apify.com/v2/acts/YOUR_USERNAME~google-search-serp-scraper/runs?token=YOUR_API_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{
    "queries": ["best running shoes", "seo rank tracker"],
    "countryCode": "us",
    "languageCode": "en",
    "resultsPerPage": 10
  }'
```

#### 2. Python (`apify-client`)

```python
from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run_input = {
    "queries": ["python programming", "apify web scraping"],
    "countryCode": "gb",
    "languageCode": "en",
    "maxPagesPerQuery": 2,
    "device": "desktop"
}

run = client.actor("YOUR_USERNAME/google-search-serp-scraper").call(run_input=run_input)

## Fetch results from dataset
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"[{item['resultType'].upper()}] {item.get('title')} - {item.get('url')}")
```

#### 3. JavaScript / Node.js (`apify-client`)

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_API_TOKEN' });

const run = await client.actor('YOUR_USERNAME/google-search-serp-scraper').call({
    queries: ['cybersecurity tools', 'cloud backup solutions'],
    countryCode: 'de',
    languageCode: 'de',
    resultsPerPage: 20,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Extracted ${items.length} SERP elements`);
```

***

### ⏰ Scheduling

To run this scraper automatically (e.g. daily for rank tracking):

1. Navigate to the Actor in **Apify Console**.
2. Click the **Schedules** tab.
3. Click **Add schedule**.
4. Set the Cron expression (e.g., `0 6 * * *` for daily at 6:00 AM UTC).
5. Specify your search queries in the Input payload.
6. Connect the run to a Webhook or Slack/Email integration to receive automated alerts!

***

### 🔒 Hard AI Overview Exclusion Guarantee

This Actor is engineered to return clean, reliable SERP search results without generative interference:

- DOM nodes matching Google AI Overviews (`[aria-label="AI Overview"]`, `[data-attrid="wa:/description"]`, `.kno-ah`, and generative text containers) are automatically purged from memory before any parser runs.
- The `excludeAiOverview` flag is locked to `true` and enforced in code.
- No third-party AI answer engines (Perplexity, Copilot, ChatGPT, Gemini) are integrated.

# Actor input Schema

## `queries` (type: `array`):

List of search queries to scrape. You can enter multiple keywords.

## `resultsPerPage` (type: `integer`):

Number of results to retrieve per page (Google num parameter, e.g., 10, 20, 30, 40, 50, 100).

## `maxPagesPerQuery` (type: `integer`):

Maximum number of pagination pages to scrape per search query.

## `countryCode` (type: `string`):

Two-letter ISO 3166-1 alpha-2 country code (e.g., 'us', 'gb', 'de', 'fr', 'jp'). Maps to Google country domain and gl parameter.

## `languageCode` (type: `string`):

Two-letter ISO 639-1 language code (e.g., 'en', 'de', 'fr', 'es'). Sets interface language and hl parameter.

## `location` (type: `string`):

Precise canonical location string (e.g., 'Austin,Texas,United States' or 'Berlin,Germany'). Encoded automatically as Google UULE.

## `device` (type: `string`):

Emulated device type. Controls headers and User-Agent.

## `includeOrganicResults` (type: `boolean`):

Extract regular organic SERP results.

## `includePaidAds` (type: `boolean`):

Extract sponsored top and bottom Google Ads.

## `includePeopleAlsoAsk` (type: `boolean`):

Extract People Also Ask (PAA) questions and answers.

## `includeRelatedSearches` (type: `boolean`):

Extract related search terms suggested by Google.

## `includeKnowledgeGraph` (type: `boolean`):

Extract Knowledge Panel / Answer Box data when available.

## `includeShoppingResults` (type: `boolean`):

Extract Google Shopping product cards (PLA) with prices and ratings.

## `includeSiteLinks` (type: `boolean`):

Extract sitelinks under organic results and ads.

## `excludeAiOverview` (type: `boolean`):

Strictly locked to true. Automatically removes and discards all Google AI Overviews and generative AI blocks from the SERP.

## `maxConcurrency` (type: `integer`):

Maximum number of parallel requests. Tuned high for optimal crawling speed.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration. Default uses purpose-built GOOGLE\_SERP group. Fallback to RESIDENTIAL if needed.

## `outputFormat` (type: `string`):

Choose output destination format.

## `debugMode` (type: `boolean`):

Enables verbose logging and saves raw HTML snapshots to Key-Value Store under DEBUG-\*.

## Actor input object example

```json
{
  "queries": [
    "best running shoes",
    "web scraping with apify"
  ],
  "resultsPerPage": 10,
  "maxPagesPerQuery": 1,
  "countryCode": "us",
  "languageCode": "en",
  "device": "desktop",
  "includeOrganicResults": true,
  "includePaidAds": true,
  "includePeopleAlsoAsk": true,
  "includeRelatedSearches": true,
  "includeKnowledgeGraph": true,
  "includeShoppingResults": true,
  "includeSiteLinks": true,
  "excludeAiOverview": true,
  "maxConcurrency": 30,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "GOOGLE_SERP"
    ]
  },
  "outputFormat": "dataset",
  "debugMode": false
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset containing extracted organic search results, paid ads, People Also Ask, related searches, knowledge graphs, and shopping results without AI Overviews.

## `resultsSummary` (type: `string`):

Comprehensive run metrics including query throughput (QPS and PPS), counts by result type, and block diagnostics stored in Key-Value Store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "best running shoes",
        "web scraping with apify"
    ],
    "excludeAiOverview": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("autoharvestor/google-search-serp-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "best running shoes",
        "web scraping with apify",
    ],
    "excludeAiOverview": True,
}

# Run the Actor and wait for it to finish
run = client.actor("autoharvestor/google-search-serp-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "best running shoes",
    "web scraping with apify"
  ],
  "excludeAiOverview": true
}' |
apify call autoharvestor/google-search-serp-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,autoharvestor/google-search-serp-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/z8DF3V8wYpuABh7Bn/builds/uwd4uyxOqQACXYdoc/openapi.json