# Google AI Overview Scraper (`s-r/google-ai-overview-scraper`) Actor

Scrape Google AI Overviews and Google AI Mode answers at scale, without a browser. This google ai overview scraper loads the Google AI Mode search page (udm=50) and calls Google's internal streaming endpoint directly, so you get the full AI-generated answer plus the exact web sources it cited.

- **URL**: https://apify.com/s-r/google-ai-overview-scraper.md
- **Developed by:** [SR](https://apify.com/s-r) (community)
- **Categories:** AI, Marketing
- **Stats:** 5 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Google AI Overview Scraper: the AI answer above the search results

This google ai overview scraper returns the AI-generated summary Google now
places above the blue links, as structured JSON. For every query you get the
full answer text, the same answer broken into headings, paragraphs and lists
with the citations attached per block, and every source page Google grounded
the answer on.

There is no official API for AI Overview content. It is also not in the search
page's HTML when you fetch it, which is why an ordinary google serp scraper
returns an empty box where the overview should be. This actor returns the
finished answer, one dataset row per query.

### What you get

- **The complete answer text** as one plain string (`answer_text`), ready for a
  spreadsheet cell, a prompt, or a diff against last week's run.
- **The answer with its structure intact** (`answer_blocks`): headings stay
  headings, bullet lists stay lists, and each block carries the citations that
  belong to it. This is what lets you attribute a specific claim to a specific
  source instead of guessing.
- **Every cited source page** (`sources`) with the site name, domain, page
  title, the snippet Google pulled, the destination URL, a favicon and a
  thumbnail where one exists.
- **The distinct domains** (`source_domains`) and their count, so a visibility
  query is a single field lookup rather than a parse.
- **Products the answer pointed at** (`products`) with title, URL, image and
  Google's own product identifiers, on shopping-shaped queries.
- **Links inside the answer body** (`links`) with Google's redirect already
  resolved to the real destination.
- **An honest present/absent flag** (`answered`, `ai_overview_present`). Google
  does not write an overview for every query. Those rows come back marked
  rather than silently empty, and you are not billed for them.

### Why scrape Google AI Overviews

The overview sits above the first organic result and answers the question in
place. When it does, the click that used to go to the page that ranked first
often does not happen at all. Rank tracking still reports position one while
the traffic behind it drops, and nothing in a rank report explains the gap.
The only way to see it is to read the overview itself and check whether your
domain is among the sources it cites.

That check has become its own discipline. Being cited by the AI answer is now
a distinct outcome from ranking, with its own winners: pages that answer a
question cleanly and get quoted, rather than pages that merely rank. The
overview also draws on sources that do not rank on page one at all, which
means a competitor invisible in your rank report can be the one Google is
quoting.

Tracking it needs the answer and its sources together. A screenshot tells you
an overview exists; it does not let you count how often a domain appears across
five hundred queries, or notice that the answer's wording changed while the
source list stayed the same. Structured output turns both into a query.

### Input

| Field | Type | Required | Default | Description |
|---|---|---|---|---|
| `queries` | array of strings | yes | — | One search query per line. |
| `country` | string | no | `us` | Two-letter country code. Sets the market the answer is written for. |
| `language` | string | no | `en` | Two-letter language code for the answer. |
| `retries` | integer | no | `3` | Extra rounds to spend when a query comes back without an overview. Range 0 to 5. |

Questions and comparisons draw an overview most reliably. A bare brand or
product name often does not get one at all; that is reported as
`answered: false` rather than retried indefinitely.

### Output

One row per query.

```json
{
  "query": "is the lego 10313 worth buying",
  "country": "us",
  "language": "en",
  "answered": true,
  "ai_overview_present": true,
  "answer_text": "Yes, the LEGO Icons Wildflower Bouquet 10313 is widely considered worth buying for its vibrant color palette, relaxing build experience, and exceptional value at a retail price of $59.99 for 939 pieces...",
  "answer_blocks": [
    {
      "type": "paragraph",
      "text": "Yes, the LEGO Icons Wildflower Bouquet 10313 is widely considered worth buying...",
      "citations": [
        { "uuid": "c396dd25", "site_name": "Amazon.com", "domains": ["amazon.com"] }
      ]
    },
    { "type": "heading", "text": "Why It Is Worth Buying", "level": 3, "citations": [] },
    { "type": "list", "items": ["Vibrant color palette", "939 pieces"], "citations": [] }
  ],
  "answer_block_count": 7,
  "sources": [
    {
      "site_name": "RTINGS.com",
      "domain": "rtings.com",
      "title": "LEGO Wildflower Bouquet 10313 Review",
      "snippet": "Giving and receiving flowers that never wilt...",
      "url": "https://www.rtings.com/...",
      "favicon": "https://...",
      "thumbnail": "https://...",
      "citation_id": "c396dd25"
    }
  ],
  "source_domains": ["amazon.com", "rebrickable.com", "reddit.com"],
  "source_count": 9,
  "links": [{ "text": "LEGO Icons Wildflower Bouquet", "url": "https://...", "kind": "product" }],
  "products": [
    { "title": "LEGO Icons Wildflower Bouquet 10313", "url": "https://...", "image": "https://...",
      "catalog_id": "9131157491774088421", "product_id": "", "merchant_id": "", "cluster_id": "" }
  ],
  "attempts": 1,
  "error": null,
  "fetched_at": "2026-09-09T15:41:02Z",
  "duration_seconds": 8.0,
  "response_bytes": 476246
}
```

### Use cases

**Answer engine optimisation.** Feed in the queries you care about and record
which domains the overview cites for each one. Run it weekly and the citation
set becomes a time series: you can see the week a competitor entered the answer,
the week you dropped out, and whether a rewrite moved you back in. `source_domains`
is the field that carries the whole analysis.

**Explaining a traffic drop that rank tracking cannot.** When impressions hold
and clicks fall, pull the overview for the affected queries. If your page is
cited, the answer is satisfying the intent in place. If it is not cited but a
competitor is, that is a different problem with a different fix. Either way you
have the evidence in an afternoon rather than a quarter.

**Competitive and category monitoring.** Run the head terms for a category and
count domain frequency across the whole set. The domains that appear again and
again are the ones Google treats as authoritative for that category, which is a
more useful target list than a rank report, because it includes sites that never
appear on page one.

**Content research.** `answer_blocks` shows the shape Google chose for an
answer: which subheadings it used, what it put in a list, what it led with.
That is a direct read on how the question is best structured, taken from the
system that decides what a good answer looks like.

### How it compares

| | This actor | Typical alternative |
|---|---|---|
| Answer structure | Headings, paragraphs and lists preserved, with per-block citations | Answer as one flat string |
| Sources | Full card per source: name, domain, title, snippet, URL, favicon, thumbnail | Domain list, or URLs only |
| Products | Google's product identifiers exposed on shopping queries | Not returned |
| No-overview queries | Returned and marked, not billed | Often a failed run |
| Start fee | $0.002 per run | Up to $0.09 per run on some listings |

Apify's own `apify/google-ai-overviews-scraper` is the category anchor with
roughly 14,700 runs; its per-result price is tiered by plan, so compare it
against your own plan's rate. Of the independent listings, `clearpath/google-ai-overview`
charges $0.09 per run before any result is returned and `johnvc/Google-AI-Overview-API`
charges a $0.01 setup event per run; both figures are from their live pricing.
This actor charges $0.002 per run plus per answer returned.

### Pricing

$0.00999 per AI Overview returned, plus a $0.002 actor start fee per run.
Queries that come back without an overview are returned in the dataset and are
not charged. All pricing is pay-per-event, so you only pay for results you
receive. There are no per-compute-unit charges.

### Limits and gotchas

- Not every query has an AI Overview. Informational and comparison queries
  usually do; navigational and bare-brand queries usually do not. `answered:
  false` with `error: "no_ai_overview"` means Google offered none, not that the
  run failed.
- The overview is regenerated per request, so wording varies slightly between
  runs even for the same query. The cited source set is far more stable than the
  prose; build tracking on `source_domains`, not on string equality of the text.
- Set `country` and `language` together. An English answer for the Dutch market
  is a different answer from a Dutch one, and mixing them produces results that
  match neither.
- Queries run a few at a time inside the run. A list of 500 is fine; expect it
  to take minutes, not seconds.
- Free Apify plans are capped at 10 rows per run. Split larger lists across runs
  or upgrade to remove the cap.
- `products` is populated only on shopping-shaped queries. It is empty for
  informational ones, which is expected rather than a miss.

### FAQ

**Can I scrape Google AI Overviews without an API key?**
Yes. Run the actor from the Apify Store or call it through the Apify API. No
Google account, key or quota is involved.

**How do I know which sites Google cites in its AI Overview?**
Read `source_domains` for the distinct list, or `sources` for the full card per
cited page including its title and the snippet Google used.

**Why did some of my queries return no AI Overview?**
Google does not write one for every query. Those rows are returned with
`answered: false` and are not billed, so a mixed query list costs only what it
returned.

**Is the AI Overview different from Google AI Mode?**
Yes, they are separate surfaces with different answers. The overview sits above
the ordinary results; AI Mode is the conversational tab. For AI Mode use the
Google AI Mode Scraper below.

**Can I track AI Overview citations for my domain over time?**
Yes. Run the same query list on a schedule and store `source_domains` per run.
The presence or absence of your domain across runs is the metric.

### Related Actors

- [Google AI Mode Scraper](https://apify.com/s-r/google-ai-mode-scraper) for the
  conversational AI tab.
- [Google AI Product Answers](https://apify.com/s-r/google-ai-product-answers)
  for AI answers about a specific product, from a barcode or model number.
- [Google Search Scraper](https://apify.com/s-r/google-serp) for the ordinary
  organic results behind the overview.

# Actor input Schema

## `queries` (type: `array`):

One search query per line. Questions and comparisons draw an AI Overview most reliably; a bare brand or product name often does not get one at all, which is reported rather than retried forever.

## `country` (type: `string`):

Two-letter country code. Sets the market the answer is written for, so 'nl' returns Dutch shops and Dutch phrasing while 'us' returns the US view of the same question.

## `language` (type: `string`):

Two-letter language code for the answer, for example en, nl, de, fr, es.

## `retries` (type: `integer`):

How many extra rounds to spend when a query comes back without an AI Overview. Each round tries fresh sessions. Three is enough for almost every query; raise it only if you see empty results on queries you know have an overview.

## Actor input object example

```json
{
  "queries": [
    "how does web scraping work"
  ],
  "country": "us",
  "language": "en",
  "retries": 3
}
```

# Actor output Schema

## `results` (type: `string`):

One row per query with the answer and its sources.

## `output` (type: `string`):

OUTPUT record with the run's counts and status flags.

## `errors` (type: `string`):

Failures with a code and a redacted message. Absent when the run had none.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "is the lego 10313 worth buying",
        "best noise cancelling headphones 2026"
    ],
    "country": "us",
    "language": "en",
    "retries": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("s-r/google-ai-overview-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": [
        "is the lego 10313 worth buying",
        "best noise cancelling headphones 2026",
    ],
    "country": "us",
    "language": "en",
    "retries": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("s-r/google-ai-overview-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "is the lego 10313 worth buying",
    "best noise cancelling headphones 2026"
  ],
  "country": "us",
  "language": "en",
  "retries": 3
}' |
apify call s-r/google-ai-overview-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,s-r/google-ai-overview-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Hx3DbvrXFydOXmpgx/builds/FhfY764gqTBch5Kfe/openapi.json