# Google AI Overview Scraper (`automation-lab/google-ai-overview-scraper`) Actor

Extract Google AI Overview answers, sections, cited source links, and domain visibility for query lists across countries and languages.

- **URL**: https://apify.com/automation-lab/google-ai-overview-scraper.md
- **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community)
- **Categories:** SEO tools, AI, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google AI Overview Scraper

Extract each **Google AI Overview** answer, its structured sections, cited source URLs, and cited domains for a supplied set of search queries.

Use the Actor to monitor whether Google shows an AI-generated answer, see which publishers receive attribution, and create repeatable visibility snapshots for SEO, GEO, AEO, content, and brand research.

The Actor returns one typed observation per query. A row remains useful when no AI Overview appears: `aiOverviewPresent` is `false` and `status` explains the outcome.

### What does Google AI Overview Scraper do?

For every query, the Actor:

1. sends a country- and language-specific Google Search request;
2. detects whether a server-rendered AI Overview is present;
3. extracts the opening summary and structured answer sections;
4. resolves and deduplicates external citation links;
5. normalizes citation hostnames;
6. checks optional target domains for citation visibility;
7. stores the observation in the run's default dataset.

This is an HTTP-first Actor. It uses Apify's specialized Google Search proxy instead of launching a browser for every query, which keeps memory and runtime bounded.

### Who is it for?

#### SEO and GEO teams

Track whether priority informational queries trigger AI answers and which domains Google cites.

#### Brand and communications teams

Add your own domain and relevant publishers to `targetDomains`, then schedule the Actor to monitor citation visibility.

#### Content strategists

Inspect the headings and statements Google uses to frame a topic. Use the cited URLs as research leads, not as a substitute for editorial review.

#### Agencies

Run separate Tasks for clients, locales, or topic groups and export each dataset to a spreadsheet, warehouse, or dashboard.

#### Developers and AI agents

Call the Actor through the Apify API or MCP and receive normalized JSON instead of maintaining Google response parsing and proxy routing.

### Why use this Actor?

- **Answer and evidence together:** each result combines normalized Overview text with the source URLs Google linked.
- **Structured sections:** headings and list statements are returned separately in `sections`.
- **Domain visibility checks:** `targetDomains` indicates which monitored sites were cited and provides matching URLs.
- **Locale controls:** country and language are explicit in every record.
- **Honest no-result rows:** an absent Overview is not presented as an extraction failure.
- **Bounded retries:** transient failures can rotate the specialized proxy session without an unlimited request loop.
- **Integration-ready output:** one consistent dataset row per query works well for scheduled comparisons.

Google decides when an AI Overview appears. The same query can produce a different answer, source set, or no Overview at another time or locale.

### What data can you extract?

| Field | Meaning |
| --- | --- |
| `query` | Submitted Google search query |
| `countryCode` | Two-letter search country |
| `languageCode` | Google interface and answer language |
| `locale` | Combined language-country label |
| `aiOverviewPresent` | Whether useful Overview text was extracted |
| `overviewText` | Normalized summary plus section content |
| `sections` | Structured headings and their statements |
| `citedSources` | Deduplicated citation title, URL, and domain objects |
| `citedDomains` | Unique cited hostnames |
| `citationCount` | Number of unique cited URLs |
| `targetDomains` | Requested domains with citation matches and URLs |
| `sourceUrl` | Canonical HTTPS Google Search URL |
| `fetchedAt` | Observation timestamp |
| `status` | `ok`, `no_overview`, `blocked`, or `error` |
| `error` | Query-level diagnostic when applicable |

All source-dependent fields are nullable in the dataset schema because Google can change or omit response components.

### How to run the Google AI Overview Scraper

1. Open the Actor in Apify Console.
2. Add one or more real questions or keywords to **Search queries**.
3. Choose a two-letter **Country code**, such as `US` or `GB`.
4. Choose a **Language code**, such as `en`, `de`, or `es`.
5. Optionally add domains you want to monitor.
6. Keep concurrency at the default for the first run.
7. Click **Start**.
8. Open the dataset to review text, citations, statuses, and monitored-domain matches.
9. Save the input as a Task if you want to schedule recurring snapshots.

A practical one-query input is:

```json
{
  "queries": ["how does solar energy work"],
  "countryCode": "US",
  "languageCode": "en",
  "targetDomains": ["energy.gov"],
  "maxConcurrency": 1,
  "maxRequestRetries": 1
}
```

### Input parameters

| Input | Type | Default | Notes |
| --- | --- | --- | --- |
| `queries` | string array | required | 1–100 unique queries; each query may be up to 400 characters |
| `countryCode` | string | `US` | two-letter ISO country code used by the proxy and Google `gl` parameter |
| `languageCode` | string | `en` | two- or three-letter language code, optionally with a region suffix |
| `targetDomains` | string array | `[]` | up to 50 hostnames; URLs are normalized to hostnames |
| `maxConcurrency` | integer | `3` | 1–5 simultaneous requests |
| `maxRequestRetries` | integer | `1` | 0–2 retries for transient request failures |

Duplicate and blank queries are removed before processing. Invalid locale values or domains fail closed before paid search requests begin.

A retry can incur another specialized proxy request, so increase `maxRequestRetries` only when transient failures justify it.

### Output example

A current US-English observation has this shape:

```json
{
  "query": "how does solar energy work",
  "countryCode": "US",
  "languageCode": "en",
  "locale": "en-US",
  "aiOverviewPresent": true,
  "overviewText": "Solar energy works by capturing light and heat from the sun and changing it into useful electricity. How Solar Panels Make Electricity Sunlight hits the panel...",
  "sections": [
    {
      "title": "How Solar Panels Make Electricity",
      "items": [
        "Sunlight hits the panel: Tiny energy particles from the sun, called photons, strike the solar cells."
      ]
    }
  ],
  "citedSources": [
    {
      "title": "Department of Energy (.gov)",
      "url": "https://www.energy.gov/cmei/systems/how-does-solar-work",
      "domain": "energy.gov"
    }
  ],
  "citedDomains": ["energy.gov", "youtube.com"],
  "citationCount": 3,
  "targetDomains": [
    {
      "domain": "energy.gov",
      "cited": true,
      "citationUrls": ["https://www.energy.gov/cmei/systems/how-does-solar-work"]
    }
  ],
  "sourceUrl": "https://www.google.com/search?q=how+does+solar+energy+work&gl=us&hl=en&num=10&pws=0",
  "fetchedAt": "2026-08-10T14:00:00.000Z",
  "status": "ok",
  "error": null
}
```

Answer text and citations are live source data and will change over time.

### How much does it cost to extract Google AI Overview results?

Pricing has two events:

- **Run start:** $0.0005 once per run.
- **Item extracted:** charged only when useful Overview text is returned. The BRONZE price is $0.0074017 per successful Overview.

The `item` event fires only for a record containing useful AI Overview text. Other observation outcomes are included under the run start event.

Approximate BRONZE examples:

| Successful Overviews | BRONZE calculation |
| ---: | --- |
| 1 | one run-start event + 1 item event |
| 10 | one run-start event + 10 item events |
| 100 | one run-start event + 100 item events |

Multiply the active BRONZE item price shown above by the number of successful Overviews, then add one run-start event. These examples assume every query returns an Overview and no retry is needed. The live Apify pricing panel is authoritative for your plan tier. Platform usage handling follows the pricing shown in Console.

### Build recurring AI visibility workflows

#### Monitor a topic portfolio

Create one Task for each stable query set. Schedule it daily or weekly, then compare `aiOverviewPresent`, `overviewText`, and `citedDomains` between run datasets.

#### Track a brand or publisher

Add the brand's domain to `targetDomains`. Filter records where `targetDomains[].cited` changes between runs.

#### Feed a content research queue

Send new cited URLs to a review table. Retain the query, locale, and timestamp so analysts understand the context in which Google surfaced each source.

#### Build an AEO/GEO dashboard

Aggregate citation counts by domain, query cluster, and date. Treat each run as an observation rather than a permanent ranking.

#### Trigger alerts responsibly

Use Apify webhooks or your automation platform to alert on meaningful changes. Confirm changes across more than one run before treating a volatile SERP response as a durable trend.

### Export and integrations

The default dataset can be downloaded as JSON, JSONL, CSV, XML, Excel, or RSS through Apify.

Common integrations include:

- Google Sheets for analyst review;
- BigQuery, Snowflake, or a data lake for historical comparisons;
- Make, Zapier, n8n, or webhooks for alerts;
- Looker Studio, Power BI, or Tableau for domain visibility dashboards;
- an LLM workflow that summarizes changed answers while preserving source links.

Each run uses its own default dataset, so scheduled Tasks do not leak stale records from earlier runs.

### Use the Apify API

Replace `<APIFY_TOKEN>` with a secret environment value. Do not commit tokens.

#### cURL

```bash
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~google-ai-overview-scraper/runs?token=<APIFY_TOKEN>&waitForFinish=300" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["what is generative engine optimization"],
    "countryCode": "US",
    "languageCode": "en"
  }'
```

#### JavaScript

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/google-ai-overview-scraper').call({
  queries: ['what is generative engine optimization'],
  countryCode: 'US',
  languageCode: 'en',
  targetDomains: ['coursera.org'],
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

#### Python

```python
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/google-ai-overview-scraper").call(run_input={
    "queries": ["how does solar energy work"],
    "countryCode": "US",
    "languageCode": "en",
    "targetDomains": ["energy.gov"],
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)
```

For asynchronous workflows, start the run without waiting and consume the dataset after the run reaches `SUCCEEDED`.

### Use with MCP and AI agents

Add the Actor to Claude Code:

```bash
claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/google-ai-overview-scraper"
```

For **Claude Desktop**, **Cursor**, and **VS Code**, add the same HTTP server URL to the client's MCP JSON configuration:

```json
{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/google-ai-overview-scraper"
    }
  }
}
```

Example prompts:

- “Check Google AI Overviews in the US for these five product-category questions and return cited domains.”
- “Run the saved solar-energy query set and flag any result that cites energy.gov.”
- “Compare the latest output with yesterday's dataset and summarize citation changes.”

An agent should preserve query, locale, timestamp, and source URLs when it summarizes the output.

### Limits and source behavior

- Google controls AI Overview eligibility; not every query returns one.
- Results can vary by country, language, time, experiments, and user context.
- The Actor observes public logged-out search behavior and does not reproduce a personal Google account.
- Up to 100 unique queries are accepted per run.
- Country availability depends on Google and the specialized search proxy.
- Google may change HTML structures. A parser mismatch is reported rather than silently inventing an answer.
- The Actor extracts the server-rendered response; it does not click “Show more” or resolve every deferred interactive variant.
- Citation titles are derived from nearby Google labels and may be less descriptive than the destination page title.
- Source pages are not downloaded, so the Actor does not verify their content.

Use multiple observations before drawing ranking or attribution conclusions.

### Failure handling and troubleshooting

#### Why is `aiOverviewPresent` false?

The normal explanation is that Google did not show an AI Overview for that query and locale. Check `status`; `no_overview` is a valid observation rather than an extraction failure.

#### Why did the run fail after producing rows?

If every query is blocked or errors, the run exits non-zero after recording query-level diagnostics. Partial runs retain useful rows and report the failure count in logs.

#### What should I do with HTTP 429 or a transient error?

Keep the default bounded retry first. Re-run later if Google or the proxy is temporarily rate-limited. Avoid raising concurrency and retries together.

#### Why does a monitored domain not match?

Enter a hostname such as `energy.gov`, not a search phrase. Subdomains count as matches for the parent domain. Redirect destinations are normalized before matching.

#### Why did the citation set change?

Google AI answers are dynamic. Confirm the locale and schedule, then compare several runs before escalating an isolated change.

### Responsible use and legality

This Actor accesses public Google Search responses. You are responsible for using the data in accordance with applicable laws, Google's terms, Apify's terms, and your organization's policies.

Do not use the output to identify or profile individuals, bypass access controls, send unsolicited messages, or make high-impact decisions without human review.

Respect copyright and attribution. AI Overview text and cited content belong to their respective owners. Store only what your use case requires, apply appropriate retention controls, and link back to original sources when publishing analysis.

This documentation is operational guidance, not legal advice.

### Related Automation Lab Actors

- [Google Search Scraper](https://apify.com/automation-lab/google-search-scraper) — collect ordinary organic results, People Also Ask, and related searches when you need broader SERP coverage.
- [Google Trends Scraper](https://apify.com/automation-lab/google-trends-scraper) — add query-interest trends to topic and content monitoring.
- [Google Autocomplete Scraper](https://apify.com/automation-lab/google-autocomplete-scraper) — expand seed topics into real Google suggestion sets before checking AI Overview coverage.

Use this Actor when AI-generated summaries and their citations are the primary record. Use the related Actors when you need organic rankings, trend history, or query discovery.

### FAQ

#### Does the Actor require my Google account?

No. It observes public logged-out search responses and does not accept Google credentials.

#### Does one query always return one AI Overview?

No. It always returns one observation row, but `aiOverviewPresent` can be false.

#### What happens when no AI Overview appears?

The Actor still records the observation. The `item` event fires only when useful Overview text was extracted.

#### Can I monitor several brands at once?

Yes. Add up to 50 domains to `targetDomains`. Every result reports each domain independently.

#### Can I schedule the Actor?

Yes. Save the input as an Apify Task and attach a schedule. Historical comparison is performed in your spreadsheet, warehouse, automation, or analysis layer.

#### Does it return organic Google results too?

No. It focuses on AI Overview answer content and citations. Use the related Google Search Scraper for organic-result coverage.

#### Can I use a custom proxy?

No input-level proxy override is exposed. The Actor intentionally uses the specialized Google Search proxy route that was validated for this response type.

#### Is output deterministic?

The schema is stable, but Google content is not. Query, locale, experiment state, and time can change the observed answer and citations.

# Actor input Schema

## `queries` (type: `array`):

Questions or keywords to check on Google. Each unique query produces one observation record, including when no AI Overview appears.

## `countryCode` (type: `string`):

Two-letter ISO country code used for Google Search geography, such as US, GB, DE, or AU.

## `languageCode` (type: `string`):

Google interface and answer language, such as en, de, es, or pt-br.

## `targetDomains` (type: `array`):

Optional domains to flag when they appear among AI Overview citations. Enter hostnames such as energy.gov without paths.

## `maxConcurrency` (type: `integer`):

Number of Google queries fetched in parallel. Keep this low for stable search-proxy sessions.

## `maxRequestRetries` (type: `integer`):

Bounded retries for transient network, rate-limit, or server failures. Each retry can incur another search-proxy request.

## Actor input object example

```json
{
  "queries": [
    "what is generative engine optimization",
    "how does solar energy work"
  ],
  "countryCode": "US",
  "languageCode": "en",
  "targetDomains": [
    "energy.gov"
  ],
  "maxConcurrency": 3,
  "maxRequestRetries": 1
}
```

# Actor output Schema

## `dataset` (type: `string`):

Structured summaries, sections, citations, monitored-domain matches, and query statuses.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "what is generative engine optimization",
        "how does solar energy work"
    ],
    "countryCode": "US",
    "languageCode": "en",
    "targetDomains": [
        "energy.gov"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("automation-lab/google-ai-overview-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "what is generative engine optimization",
        "how does solar energy work",
    ],
    "countryCode": "US",
    "languageCode": "en",
    "targetDomains": ["energy.gov"],
}

# Run the Actor and wait for it to finish
run = client.actor("automation-lab/google-ai-overview-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "what is generative engine optimization",
    "how does solar energy work"
  ],
  "countryCode": "US",
  "languageCode": "en",
  "targetDomains": [
    "energy.gov"
  ]
}' |
apify call automation-lab/google-ai-overview-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,automation-lab/google-ai-overview-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/8nhMUfqHbiSDxcSoP/builds/0VieeVZlu6avwmlPa/openapi.json