Google AI Overviews Scraper — AIO Citations & Brand Visibility

Scrape Google AI Overview (AIO) for any query. Returns the AI-generated answer text, every cited source (domain, URL, title, position), and optional brand-match detection to see whether your domain is cited — and at what rank. Multi-country, desktop/mobile, MCP-ready for Claude and other AI agents.

Built for SEO/AEO agencies, in-house GEO (Generative Engine Optimization) teams, and brand monitors who need a clean, programmatic way to track visibility in Google's zero-click AI Overview era.

Reliability: Camoufox + Apify residential is the default scrape path. When Google captcha blocks the request, we transparently fall back to our managed SerpApi infrastructure to deliver the result — no setup required from you. Bring your own SerpApi key (serpApiKey input) to pay only the standard $0.015 rate on every query.

What does Google AI Overviews Scraper do?

Sends one or many Google search queries against google.com/search, waits for the AI Overview block to render, then parses:

• The full AIO answer text and individual text blocks
• Every cited source: domain, full URL (Google /url?q= redirects decoded), citation position (1-based), title shown on the citation card
• Optional brand match against your targetDomain (or comma-separated multiple domains)
• Country, language, device — all configurable per run

The scraper uses a hybrid path: a Camoufox (stealth Firefox) browser scrape over Apify residential proxies as the primary, with an optional SerpApi BYOK fallback when a query fails after retries. You only pay for the SerpApi side if you bring your own key — most queries succeed on Camoufox.

Field	Type	Example
query	string	"best CRM software 2026"
aioPresent	boolean	true
aioText	string | null	"The best CRM software in 2026 includes HubSpot…"
aioTextBlocks	string[]	["Block 1 text…", "Block 2 text…"]
sources[]	array	see below
sources[].position	integer	1
sources[].domain	string	"hubspot.com"
sources[].url	string	"https://www.hubspot.com/products/crm"
sources[].title	string | null	"HubSpot CRM — Free Forever"
sourceCount	integer	8
uniqueDomainCount	integer	7
brandMatch.matched	boolean	true
brandMatch.position	integer | null	3
brandMatch.citedUrl	string | null	"https://apify.com/blog/best-crm"
brandMatch.matchCount	integer	2
source	"camoufox" | "serpapi"	"camoufox"
scrapedAt	ISO 8601	"2026-05-29T14:32:11.000Z"

How much does it cost to scrape Google AI Overviews?

• Actor start: $0.00005 per GB RAM (default 2 GB → $0.0001 per run start)
• AI Overview query checked: $0.015 per query — when Camoufox succeeds, OR when you bring your own SerpApi key (BYOK) for the fallback.
• AI Overview query (managed SerpApi fallback): $0.10 per query — when our Camoufox is captcha-blocked and you didn't provide a SerpApi key, we use our managed SerpApi infrastructure to still deliver the result. Covers SerpApi API cost + reliability margin.
• Brand match detected: $0.005 bonus only when targetDomain matches one or more citations

Typical cost (no SerpApi key): Mix of $0.015 (Camoufox success) and $0.10 (managed SerpApi fallback). Real-world Google captcha rate on Apify residential is ~50%, so blended ≈ $0.057/query. A 100-keyword brand-visibility report runs about $5.70.

Typical cost (with your own SerpApi key): Flat $0.015 per query regardless of which path delivers — you pay SerpApi directly for your queries (free tier covers 100/mo at serpapi.com). A 100-keyword report runs about $1.50 on our side + your SerpApi spend.

Never charged: Queries that fail on both paths (rare). You only pay for results we deliver.

Why use Google AI Overviews Scraper?

• Zero-click visibility is the new SERP. AIO replaces organic clicks for informational and how-to queries. If you're not cited, you're invisible.
• Built for AEO/GEO reporting. Brand match per query → share-of-voice dashboards.
• MCP-ready. AI agents (Claude, ChatGPT, Cursor) can call this as a tool — single query in, structured JSON out.
• Hybrid reliability. Camoufox + residential proxies handle the majority of queries. SerpApi BYOK kicks in only when needed.
• Multi-country, multi-device. AIO varies by country and device — we surface both axes.

How to use Google AI Overviews Scraper

1. Open the Apify Console and create a new run.
2. Fill Search queries — one per line, max 50.
3. (Optional) Set Brand domain to detect — e.g., yourbrand.com.
4. Pick Country code and Language code (defaults: us / en).
5. Choose Device — desktop or mobile.
6. (Optional) Paste a SerpApi key for fallback.
7. Click Save & Start. Results appear in the dataset; the SUMMARY key holds run-level counters.

Input

{
  "queries": [
    "best project management software 2026",
    "how does generative AI work",
    "top SEO tools for agencies"
  ],
  "targetDomain": "apify.com",
  "countryCode": "us",
  "languageCode": "en",
  "device": "desktop"
}

See the Input tab in Apify Console for the form, including SerpApi BYOK and proxy overrides.

Output

Dataset items can be downloaded as JSON, CSV, Excel, RSS, or HTML. Example row:

{
  "query": "best project management software 2026",
  "countryCode": "us",
  "languageCode": "en",
  "device": "desktop",
  "serpUrl": "https://www.google.com/search?q=best+project+management+software+2026&hl=en&gl=us&num=10",
  "aioPresent": true,
  "aioText": "Top project management tools in 2026 include Asana, Monday.com, ClickUp, and Notion — each with distinct strengths for team size and workflow complexity…",
  "aioTextBlocks": [
    "Top project management tools in 2026 include Asana, Monday.com, ClickUp, and Notion…",
    "For small teams under 10 people, ClickUp's free tier and Notion's project templates are commonly recommended…"
  ],
  "sources": [
    { "position": 1, "domain": "asana.com", "url": "https://asana.com/guide", "title": "Project Management Guide — Asana" },
    { "position": 2, "domain": "monday.com", "url": "https://monday.com/blog/project-management/", "title": "Monday.com — Best for Teams" },
    { "position": 3, "domain": "clickup.com", "url": "https://clickup.com/blog/best-pm-tools/", "title": "ClickUp Best PM Tools 2026" }
  ],
  "sourceCount": 3,
  "uniqueDomainCount": 3,
  "brandMatch": { "targetDomain": "apify.com", "matched": false, "position": null, "citedUrl": null, "matchCount": 0 },
  "source": "camoufox",
  "scrapedAt": "2026-05-29T14:32:11.000Z",
  "warnings": []
}

Use with AI agents (MCP)

This actor is MCP-first. Connect it from Claude or another MCP-aware client:

https://mcp.apify.com?tools=khadinakbar/google-ai-overviews-scraper

Then ask the agent: "Check the Google AI Overview for 'best CRM software 2026' and tell me if apify.com is cited." The agent will call the actor, parse the JSON, and answer with citation rank.

# Direct Apify API call
from apify_client import ApifyClient
client = ApifyClient("YOUR_TOKEN")
run = client.actor("khadinakbar/google-ai-overviews-scraper").call(run_input={
    "queries": ["best CRM software 2026", "top SEO tools 2026"],
    "targetDomain": "apify.com",
    "countryCode": "us",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["query"], "→ AIO:", item["aioPresent"], "match:", item["brandMatch"])

// Node.js
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('khadinakbar/google-ai-overviews-scraper').call({
  queries: ['best CRM software 2026'],
  targetDomain: 'apify.com',
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0]);

Tips & advanced options

• AIO doesn't render for every query. Google shows AIO mainly on informational, how-to, and comparison queries. Commercial, navigational, and ambiguous queries often skip it. aioPresent: false is a valid result — you're still charged for the check (this is industry-standard for AIO trackers).
• Use device: "mobile" for B2C queries. Google sometimes shows different AIO content on mobile.
• Bring a SerpApi key for ~99% reliability. Camoufox + residential gets us into the high 90s on most queries; SerpApi handles the long tail.
• Multi-domain brand match. Pass "apify.com,docs.apify.com,blog.apify.com" to match any of them.
• Schedule daily/weekly runs via Apify Schedules for citation-trend dashboards.

Portfolio cross-sell

This actor pairs well with our other AI-search and SEO tools:

• khadinakbar/ai-search-brand-monitor — track brand mentions across Perplexity, ChatGPT, Claude, Gemini
• khadinakbar/ai-search-visibility-tracker — track domain citations across AI engines
• khadinakbar/scrape-google-serp — full Google SERP scraper
• khadinakbar/google-trends-scraper — Google Trends data
• khadinakbar/google-news-scraper — Google News articles

FAQ

Q: Does AIO render for every query? A: No. AIO appears on a subset of queries — mainly informational, how-to, and comparison. Commercial and navigational queries often skip it. aioPresent: false is a normal result.

Q: Why is brand match null for some rows? A: Brand match is computed only when targetDomain is set in input. Leave it unset to skip matching.

Q: How accurate is the citation parsing? A: We use structural selectors (heading text + data attributes) plus link extraction inside the AIO container. Google rotates CSS class names frequently — structural anchors stay stable. If you spot drift, open an issue with the affected query.

Q: Do I need a SerpApi key? A: No — we provide a managed SerpApi fallback automatically when Google captcha blocks. You'll just pay $0.10/query for those queries instead of $0.015. Bring your own SerpApi key (free tier 100/mo at serpapi.com) to get the flat $0.015 rate on every query.

Q: What's the difference between the $0.015 and $0.10 events? A: $0.015 = Camoufox direct scrape OR your own SerpApi key. $0.10 = our managed SerpApi key (used only when Camoufox is captcha-blocked and you didn't bring a key). Same data quality; different cost path.

Q: Why $0.015/query even when AIO is absent? A: You're paying for the check — proxy bandwidth, browser CPU, parse time — not for the AIO hit. Industry-standard pricing across all AIO trackers.

Q: Can I run this on a schedule? A: Yes. Use Apify Schedules to run weekly/daily citation-trend reports.

Legal disclaimer

This actor scrapes publicly available Google search result pages. It does not bypass authentication, paywalls, or access private data. You are responsible for ensuring your use complies with Google's Terms of Service and your local laws. The actor enforces residential proxies and rate-limited concurrency to minimize impact on Google's infrastructure. No personal data is collected beyond what Google publishes in its public AI Overview citations.

Support & feedback

Open an issue on the Apify Store Issues tab. We typically respond within 24h.