Google AI Overview Scraper

Extract each Google AI Overview answer, its structured sections, cited source URLs, and cited domains for a supplied set of search queries.

Use the Actor to monitor whether Google shows an AI-generated answer, see which publishers receive attribution, and create repeatable visibility snapshots for SEO, GEO, AEO, content, and brand research.

The Actor returns one typed observation per query. A row remains useful when no AI Overview appears: aiOverviewPresent is false and status explains the outcome.

What does Google AI Overview Scraper do?

For every query, the Actor:

1. sends a country- and language-specific Google Search request;
2. detects whether a server-rendered AI Overview is present;
3. extracts the opening summary and structured answer sections;
4. resolves and deduplicates external citation links;
5. normalizes citation hostnames;
6. checks optional target domains for citation visibility;
7. stores the observation in the run's default dataset.

This is an HTTP-first Actor. It uses Apify's specialized Google Search proxy instead of launching a browser for every query, which keeps memory and runtime bounded.

Who is it for?

SEO and GEO teams

Track whether priority informational queries trigger AI answers and which domains Google cites.

Brand and communications teams

Add your own domain and relevant publishers to targetDomains, then schedule the Actor to monitor citation visibility.

Content strategists

Inspect the headings and statements Google uses to frame a topic. Use the cited URLs as research leads, not as a substitute for editorial review.

Agencies

Run separate Tasks for clients, locales, or topic groups and export each dataset to a spreadsheet, warehouse, or dashboard.

Developers and AI agents

Call the Actor through the Apify API or MCP and receive normalized JSON instead of maintaining Google response parsing and proxy routing.

Why use this Actor?

• Answer and evidence together: each result combines normalized Overview text with the source URLs Google linked.
• Structured sections: headings and list statements are returned separately in sections.
• Domain visibility checks: targetDomains indicates which monitored sites were cited and provides matching URLs.
• Locale controls: country and language are explicit in every record.
• Honest no-result rows: an absent Overview is not presented as an extraction failure.
• Bounded retries: transient failures can rotate the specialized proxy session without an unlimited request loop.
• Integration-ready output: one consistent dataset row per query works well for scheduled comparisons.

Google decides when an AI Overview appears. The same query can produce a different answer, source set, or no Overview at another time or locale.

What data can you extract?

Field	Meaning
query	Submitted Google search query
countryCode	Two-letter search country
languageCode	Google interface and answer language
locale	Combined language-country label
aiOverviewPresent	Whether useful Overview text was extracted
overviewText	Normalized summary plus section content
sections	Structured headings and their statements
citedSources	Deduplicated citation title, URL, and domain objects
citedDomains	Unique cited hostnames
citationCount	Number of unique cited URLs
targetDomains	Requested domains with citation matches and URLs
sourceUrl	Canonical HTTPS Google Search URL
fetchedAt	Observation timestamp
status	ok, no_overview, blocked, or error
error	Query-level diagnostic when applicable

All source-dependent fields are nullable in the dataset schema because Google can change or omit response components.

How to run the Google AI Overview Scraper

1. Open the Actor in Apify Console.
2. Add one or more real questions or keywords to Search queries.
3. Choose a two-letter Country code, such as US or GB.
4. Choose a Language code, such as en, de, or es.
5. Optionally add domains you want to monitor.
6. Keep concurrency at the default for the first run.
7. Click Start.
8. Open the dataset to review text, citations, statuses, and monitored-domain matches.
9. Save the input as a Task if you want to schedule recurring snapshots.

A practical one-query input is:

{
  "queries": ["how does solar energy work"],
  "countryCode": "US",
  "languageCode": "en",
  "targetDomains": ["energy.gov"],
  "maxConcurrency": 1,
  "maxRequestRetries": 1
}

Input parameters

Input	Type	Default	Notes
queries	string array	required	1–100 unique queries; each query may be up to 400 characters
countryCode	string	US	two-letter ISO country code used by the proxy and Google gl parameter
languageCode	string	en	two- or three-letter language code, optionally with a region suffix
targetDomains	string array	[]	up to 50 hostnames; URLs are normalized to hostnames
maxConcurrency	integer	3	1–5 simultaneous requests
maxRequestRetries	integer	1	0–2 retries for transient request failures

Duplicate and blank queries are removed before processing. Invalid locale values or domains fail closed before paid search requests begin.

A retry can incur another specialized proxy request, so increase maxRequestRetries only when transient failures justify it.

Output example

A current US-English observation has this shape:

{
  "query": "how does solar energy work",
  "countryCode": "US",
  "languageCode": "en",
  "locale": "en-US",
  "aiOverviewPresent": true,
  "overviewText": "Solar energy works by capturing light and heat from the sun and changing it into useful electricity. How Solar Panels Make Electricity Sunlight hits the panel...",
  "sections": [
    {
      "title": "How Solar Panels Make Electricity",
      "items": [
        "Sunlight hits the panel: Tiny energy particles from the sun, called photons, strike the solar cells."
      ]
    }
  ],
  "citedSources": [
    {
      "title": "Department of Energy (.gov)",
      "url": "https://www.energy.gov/cmei/systems/how-does-solar-work",
      "domain": "energy.gov"
    }
  ],
  "citedDomains": ["energy.gov", "youtube.com"],
  "citationCount": 3,
  "targetDomains": [
    {
      "domain": "energy.gov",
      "cited": true,
      "citationUrls": ["https://www.energy.gov/cmei/systems/how-does-solar-work"]
    }
  ],
  "sourceUrl": "https://www.google.com/search?q=how+does+solar+energy+work&gl=us&hl=en&num=10&pws=0",
  "fetchedAt": "2026-08-10T14:00:00.000Z",
  "status": "ok",
  "error": null
}

Answer text and citations are live source data and will change over time.

How much does it cost to extract Google AI Overview results?

Pricing has two events:

• Run start: $0.0005 once per run.
• Item extracted: charged only when useful Overview text is returned. The BRONZE price is $0.0074017 per successful Overview.

The item event fires only for a record containing useful AI Overview text. Other observation outcomes are included under the run start event.

Approximate BRONZE examples:

Successful Overviews	BRONZE calculation
1	one run-start event + 1 item event
10	one run-start event + 10 item events
100	one run-start event + 100 item events

Multiply the active BRONZE item price shown above by the number of successful Overviews, then add one run-start event. These examples assume every query returns an Overview and no retry is needed. The live Apify pricing panel is authoritative for your plan tier. Platform usage handling follows the pricing shown in Console.

Build recurring AI visibility workflows

Monitor a topic portfolio

Create one Task for each stable query set. Schedule it daily or weekly, then compare aiOverviewPresent, overviewText, and citedDomains between run datasets.

Track a brand or publisher

Add the brand's domain to targetDomains. Filter records where targetDomains[].cited changes between runs.

Feed a content research queue

Send new cited URLs to a review table. Retain the query, locale, and timestamp so analysts understand the context in which Google surfaced each source.

Build an AEO/GEO dashboard

Aggregate citation counts by domain, query cluster, and date. Treat each run as an observation rather than a permanent ranking.

Trigger alerts responsibly

Use Apify webhooks or your automation platform to alert on meaningful changes. Confirm changes across more than one run before treating a volatile SERP response as a durable trend.

Export and integrations

The default dataset can be downloaded as JSON, JSONL, CSV, XML, Excel, or RSS through Apify.

Common integrations include:

• Google Sheets for analyst review;
• BigQuery, Snowflake, or a data lake for historical comparisons;
• Make, Zapier, n8n, or webhooks for alerts;
• Looker Studio, Power BI, or Tableau for domain visibility dashboards;
• an LLM workflow that summarizes changed answers while preserving source links.

Each run uses its own default dataset, so scheduled Tasks do not leak stale records from earlier runs.

Use the Apify API

Replace <APIFY_TOKEN> with a secret environment value. Do not commit tokens.

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~google-ai-overview-scraper/runs?token=<APIFY_TOKEN>&waitForFinish=300" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["what is generative engine optimization"],
    "countryCode": "US",
    "languageCode": "en"
  }'

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/google-ai-overview-scraper').call({
  queries: ['what is generative engine optimization'],
  countryCode: 'US',
  languageCode: 'en',
  targetDomains: ['coursera.org'],
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("automation-lab/google-ai-overview-scraper").call(run_input={
    "queries": ["how does solar energy work"],
    "countryCode": "US",
    "languageCode": "en",
    "targetDomains": ["energy.gov"],
})
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)

For asynchronous workflows, start the run without waiting and consume the dataset after the run reaches SUCCEEDED.

Use with MCP and AI agents

Add the Actor to Claude Code:

claude mcp add --transport http apify \
  "https://mcp.apify.com?tools=automation-lab/google-ai-overview-scraper"

For Claude Desktop, Cursor, and VS Code, add the same HTTP server URL to the client's MCP JSON configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=automation-lab/google-ai-overview-scraper"
    }
  }
}

Example prompts:

• “Check Google AI Overviews in the US for these five product-category questions and return cited domains.”
• “Run the saved solar-energy query set and flag any result that cites energy.gov.”
• “Compare the latest output with yesterday's dataset and summarize citation changes.”

An agent should preserve query, locale, timestamp, and source URLs when it summarizes the output.

Limits and source behavior

• Google controls AI Overview eligibility; not every query returns one.
• Results can vary by country, language, time, experiments, and user context.
• The Actor observes public logged-out search behavior and does not reproduce a personal Google account.
• Up to 100 unique queries are accepted per run.
• Country availability depends on Google and the specialized search proxy.
• Google may change HTML structures. A parser mismatch is reported rather than silently inventing an answer.
• The Actor extracts the server-rendered response; it does not click “Show more” or resolve every deferred interactive variant.
• Citation titles are derived from nearby Google labels and may be less descriptive than the destination page title.
• Source pages are not downloaded, so the Actor does not verify their content.

Use multiple observations before drawing ranking or attribution conclusions.

Failure handling and troubleshooting

Why is aiOverviewPresent false?

The normal explanation is that Google did not show an AI Overview for that query and locale. Check status; no_overview is a valid observation rather than an extraction failure.

Why did the run fail after producing rows?

If every query is blocked or errors, the run exits non-zero after recording query-level diagnostics. Partial runs retain useful rows and report the failure count in logs.

What should I do with HTTP 429 or a transient error?

Keep the default bounded retry first. Re-run later if Google or the proxy is temporarily rate-limited. Avoid raising concurrency and retries together.

Why does a monitored domain not match?

Enter a hostname such as energy.gov, not a search phrase. Subdomains count as matches for the parent domain. Redirect destinations are normalized before matching.

Why did the citation set change?

Google AI answers are dynamic. Confirm the locale and schedule, then compare several runs before escalating an isolated change.

Responsible use and legality

This Actor accesses public Google Search responses. You are responsible for using the data in accordance with applicable laws, Google's terms, Apify's terms, and your organization's policies.

Do not use the output to identify or profile individuals, bypass access controls, send unsolicited messages, or make high-impact decisions without human review.

Respect copyright and attribution. AI Overview text and cited content belong to their respective owners. Store only what your use case requires, apply appropriate retention controls, and link back to original sources when publishing analysis.

This documentation is operational guidance, not legal advice.

Related Automation Lab Actors

• Google Search Scraper — collect ordinary organic results, People Also Ask, and related searches when you need broader SERP coverage.
• Google Trends Scraper — add query-interest trends to topic and content monitoring.
• Google Autocomplete Scraper — expand seed topics into real Google suggestion sets before checking AI Overview coverage.

Use this Actor when AI-generated summaries and their citations are the primary record. Use the related Actors when you need organic rankings, trend history, or query discovery.

FAQ

Does the Actor require my Google account?

No. It observes public logged-out search responses and does not accept Google credentials.

Does one query always return one AI Overview?

No. It always returns one observation row, but aiOverviewPresent can be false.

What happens when no AI Overview appears?

The Actor still records the observation. The item event fires only when useful Overview text was extracted.

Can I monitor several brands at once?

Yes. Add up to 50 domains to targetDomains. Every result reports each domain independently.

Can I schedule the Actor?

Yes. Save the input as an Apify Task and attach a schedule. Historical comparison is performed in your spreadsheet, warehouse, automation, or analysis layer.

Does it return organic Google results too?

No. It focuses on AI Overview answer content and citations. Use the related Google Search Scraper for organic-result coverage.

Can I use a custom proxy?

No input-level proxy override is exposed. The Actor intentionally uses the specialized Google Search proxy route that was validated for this response type.

Is output deterministic?

The schema is stable, but Google content is not. Query, locale, experiment state, and time can change the observed answer and citations.