# Google AI Overview Scraper (`egsand/google-ai-overview-scraper`) Actor

- **URL**: https://apify.com/egsand/google-ai-overview-scraper.md
- **Developed by:** [Enrique G.](https://apify.com/egsand) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $8.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Google AI Overview & Gemini Search Scraper

Extract real-time Google AI Overviews (Gemini-powered search summaries), citations, linked sources, and SERP rankings at scale. Monitor your brand visibility in generative search and dominate Generative Engine Optimization (GEO).

***

### What is Google AI Overview & Gemini Search Scraper?

When users search on Google, Google frequently generates an **AI Overview** powered by Gemini right above organic search results. These synthesized answers prominently cite selected websites, shaping search traffic and user decisions before anyone clicks on traditional organic listings.

The **Google AI Overview & Gemini Search Scraper** monitors and extracts these generative summaries. It captures the complete synthesized answer, every cited source link, domain attributions, and organic rankings for any keyword or country.

***

### Why Monitor Google AI Overviews?

- **Generative Engine Optimization (GEO / AEO):** Track whether your brand or products appear in Google's AI-generated summaries for high-value search queries.
- **Citation & Backlink Intelligence:** Discover which authority websites Google cites as sources for industry topics.
- **Competitor Tracking:** Identify competitors who are being recommended by Gemini in AI summaries and reverse-engineer their content visibility.
- **Comprehensive Coverage:** Handle searches where AI Overviews are present, as well as searches where Google serves traditional results, ensuring zero data loss.
- **Global Localization:** Track AI Overviews across different countries and languages worldwide.

***

### Key Features

- **Gemini Synthesis Extraction:** Captures full AI-generated answers, including formatted text, bullet lists, and paragraphs.
- **Source & Citation Mapping:** Extracts every external URL, domain name, snippet, and position cited inside the AI Overview.
- **Smart Status Detection:** Intelligently identifies whether an AI Overview was generated (`GENERATED`), not present for the query (`NOT_PRESENT`), or available via manual generation button (`GENERATE_BUTTON_FOUND`).
- **Organic Fallback Results:** Automatically collects organic SERP results (titles, URLs, snippets) alongside or as a fallback when AI Overviews are not triggered.
- **Flexible Search Modes:** Pass simple keyword queries or exact Google search URLs with custom search parameters.
- **Export Anywhere:** Download structured data in JSON, CSV, Excel, or consume it directly via API, Webhooks, or cloud integrations.

***

### Input Configuration

Configure your search run through the visual dashboard or by passing a JSON payload:

| Field | Type | Required | Description | Default |
|---|---|---|---|---|
| `queries` | Array of strings | No\* | List of keywords or questions to search. | `["what is generative engine optimization"]` |
| `searchUrls` | Array of strings | No\* | Direct Google search URLs for customized filtering. | `[]` |
| `countryCode` | String | No | Two-letter country code (e.g., `US`, `GB`, `ES`, `DE`, `FR`, `MX`, `CA`). | `"US"` |
| `languageCode` | String | No | Language code for search interface and results (e.g., `en`, `es`, `fr`, `de`). | `"en"` |
| `includeOrganicResults` | Boolean | No | Include organic search results alongside or as fallback. | `true` |
| `maxOrganicResults` | Integer | No | Maximum number of organic SERP results to extract per query. | `10` |
| `proxyConfiguration` | Object | No | Proxy settings. Residential proxies are strongly recommended. | Apify Residential |
| `maxConcurrency` | Integer | No | Maximum concurrent search tabs. | `2` |

*\* Note: Provide at least one search query in `queries` or one URL in `searchUrls`.*

#### Example Input

```json
{
  "queries": [
    "what is generative engine optimization",
    "best crm software for startups"
  ],
  "countryCode": "US",
  "languageCode": "en",
  "includeOrganicResults": true,
  "maxOrganicResults": 5,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

***

### Output Data Structure

Each processed search query yields a structured record in the dataset with comprehensive metadata:

- **`query`**: The searched keyword or question.
- **`searchUrl`**: The direct Google search URL executed.
- **`countryCode`**: Target country code.
- **`languageCode`**: Target language code.
- **`scrapedAt`**: ISO timestamp of the scraping run.
- **`hasAiOverview`**: `true` if a Gemini AI Overview was found; `false` otherwise.
- **`aiOverviewStatus`**:
  - `GENERATED`: AI Overview was successfully generated and captured.
  - `NOT_PRESENT`: Query did not trigger an AI Overview (common for navigational or transactional searches).
  - `GENERATE_BUTTON_FOUND`: An interactive prompt to generate an overview was displayed.
- **`aiOverview`**: *(Present when `hasAiOverview` is true)*
  - `text`: Plain text representation of the AI answer.
  - `markdownText`: Structured markdown text including bullet lists and sections.
  - `citations`: List of cited sources with `title`, `url`, `domain`, and `position`.
- **`citationsCount`**: Number of distinct cited sources.
- **`organicResults`**: List of organic search results (title, URL, domain, snippet, rank).

***

#### Example Output: Search with AI Overview

```json
{
  "query": "what is generative engine optimization",
  "searchUrl": "https://www.google.com/search?q=what+is+generative+engine+optimization&hl=en&gl=us",
  "countryCode": "US",
  "languageCode": "en",
  "scrapedAt": "2026-11-02T12:00:00.000Z",
  "hasAiOverview": true,
  "aiOverviewStatus": "GENERATED",
  "overviewTextSummary": "Generative Engine Optimization (GEO) is the practice of optimizing digital content...",
  "citationsCount": 3,
  "aiOverview": {
    "text": "Generative Engine Optimization (GEO) is the practice of optimizing digital content to increase visibility in AI-generated search engines and answer engines such as Google AI Overviews and ChatGPT Search. Key strategies include citing authoritative sources, using direct language, and structuring data with clear schema markup.",
    "markdownText": "Generative Engine Optimization (GEO) is the practice of optimizing digital content to increase visibility in AI-generated search engines...\n\n- Cite authoritative sources\n- Use direct, factual language\n- Structure content with schema markup",
    "citations": [
      {
        "position": 1,
        "title": "GEO: Generative Engine Optimization Guide",
        "url": "https://example.com/insights/geo-guide",
        "domain": "example.com",
        "snippet": "Learn how generative engines choose sources and synthesize answers."
      },
      {
        "position": 2,
        "title": "Understanding AI Overviews in Search",
        "url": "https://industry-analytics.org/ai-search",
        "domain": "industry-analytics.org",
        "snippet": "An analysis of citation patterns in Google AI summaries."
      }
    ]
  },
  "organicResults": [
    {
      "position": 1,
      "title": "What is GEO? Complete Beginner's Guide",
      "url": "https://example.com/what-is-geo",
      "domain": "example.com",
      "snippet": "Discover the fundamentals of optimizing for AI search models."
    }
  ]
}
```

***

#### Example Output: Search without AI Overview

```json
{
  "query": "buy wireless mechanical keyboard",
  "searchUrl": "https://www.google.com/search?q=buy+wireless+mechanical+keyboard&hl=en&gl=us",
  "countryCode": "US",
  "languageCode": "en",
  "scrapedAt": "2026-11-02T12:01:00.000Z",
  "hasAiOverview": false,
  "aiOverviewStatus": "NOT_PRESENT",
  "overviewTextSummary": null,
  "citationsCount": 0,
  "organicResults": [
    {
      "position": 1,
      "title": "Top Mechanical Keyboards 2026",
      "url": "https://techstore.com/keyboards",
      "domain": "techstore.com",
      "snippet": "Browse premium wireless mechanical keyboards with fast delivery."
    }
  ]
}
```

***

### Best Practices & Proxy Tips

1. **Use Residential Proxies:** Google enforces strict bot detection and IP filtering. Selecting `RESIDENTIAL` proxies in the proxy configuration ensures continuous access and high success rates.
2. **Targeting Accuracy:** Match your `countryCode` with the location of your proxies for localized search results (e.g., US proxies with `countryCode: "US"`).
3. **Pacing:** Keep concurrency between 1 and 3 concurrent requests to ensure smooth execution without rate limits.

# Actor input Schema

## `queries` (type: `array`):

List of keywords or questions to search on Google. Example: 'best cloud storage for small business' or 'symptoms of dehydration'.

## `searchUrls` (type: `array`):

Optional list of full Google search URLs. Use this if you want to apply custom Google query parameters, advanced operators, or specific filters.

## `countryCode` (type: `string`):

Two-letter country code (ISO 3166-1 alpha-2) used for localization and proxy routing (e.g. 'US', 'GB', 'ES', 'DE', 'FR', 'MX', 'CA').

## `languageCode` (type: `string`):

Language code for search interface and results (e.g., 'en' for English, 'es' for Spanish, 'de' for German, 'fr' for French).

## `includeOrganicResults` (type: `boolean`):

Whether to extract organic search results alongside or as a fallback when an AI Overview is not triggered.

## `maxOrganicResults` (type: `integer`):

Maximum number of organic search results to extract per query.

## `proxyConfiguration` (type: `object`):

Residential proxies are strongly recommended to ensure consistent access to Google AI Overviews and bypass anti-bot challenges.

## `maxConcurrency` (type: `integer`):

Maximum number of concurrent search requests. Keep it at 1 on 1GB containers to avoid memory overload.

## Actor input object example

```json
{
  "queries": [
    "what is generative engine optimization",
    "how does quantum computing work"
  ],
  "countryCode": "US",
  "languageCode": "en",
  "includeOrganicResults": true,
  "maxOrganicResults": 10,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  },
  "maxConcurrency": 1
}
```

# Actor output Schema

## `overview` (type: `string`):

Dataset containing the scraped Gemini AI Overviews, cited sources, and SERP items.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "what is generative engine optimization",
        "how does quantum computing work"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("egsand/google-ai-overview-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": [
        "what is generative engine optimization",
        "how does quantum computing work",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("egsand/google-ai-overview-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "what is generative engine optimization",
    "how does quantum computing work"
  ]
}' |
apify call egsand/google-ai-overview-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,egsand/google-ai-overview-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/iCd2fnYmudeoRFODZ/builds/7ZZZr0aIGqtgyW2HA/openapi.json