Google AI Overview Scraper – GEO, Sources & Citations

Track what Google's AI says about your brand — and who it cites.

This actor runs any list of search queries on Google and extracts the AI Overview answer directly from the results page. If no AI Overview is shown, it automatically switches to Google's AI Mode and captures the answer there. For every query you get the full response as text and Markdown, the cited sources (URL, title, domain) and an aggregated list of cited domains — the core metric for Generative Engine Optimization (GEO).

Use it to:

• Monitor AI search visibility — is your brand mentioned and cited in Google's AI answers?
• Track competitors — which domains does Google cite for your money keywords?
• GEO & SEO reporting — measure AI Overview coverage across hundreds of keywords
• Content research — see exactly what Google's AI summarizes for a topic

Features

• Batch processing: analyze many prompts in one run
• Extracts the full (expanded) AI Overview text as plain text and markdown
• AI Mode fallback: if no AI Overview is shown by default, switches to Google's AI Mode tab
• Extracts all cited sources ({ position, text, url, domain }) plus aggregated sourceDomains
• Detects prompts that do not trigger any AI answer (answerSource: null)
• Language (hl) and country (gl) targeting
• Residential proxies with selectable country, stealth browsing via Camoufox
• Captcha handled as a real failure (retries with a fresh session; success: false if persistent)

Input

Field	Type	Description
prompts	array (required)	Search queries to analyze (alias: queries)
languageCode	string	Google UI language (hl), e.g. en, de (default en)
countryCode	string	Result country (gl), e.g. us, de (optional)
aiModeFallback	boolean	If no AI Overview is shown by default, switch to Google's AI Mode tab and extract the answer there (default true)
maxConcurrency	integer	How many browser sessions to run in parallel (default 3; higher = faster, needs more memory)
maxRequestRetries	integer	Retries per session on block/captcha — fresh browser + IP each retry; already-answered queries are skipped (default 3)
promptsPerSession	integer	Queries per browser before a fresh browser/IP (default 5). The cold start is the risky moment; an established session handles follow-up searches fine, and a mid-session block safely retries only the remaining queries
blockResources	boolean	Block images/fonts/media to cut proxy traffic (biggest cost). JS/CSS/XHR kept, so extraction is unaffected (default true)
proxyCountry	string	Residential proxy country; empty (default) auto-matches the search country (gl), else US

Output

One dataset item per prompt. The schema is aligned with the ChatGPT Prompt Actor (prompt, success, responseText, responseMarkdown, sources, sourceDomains, …) so results from both actors combine easily.

• prompt: the search query
• success: true if the prompt was processed, false on failure (e.g. persistent captcha)
• error: null on success, otherwise an error marker (CAPTCHA, REQUEST_FAILED_AFTER_RETRIES, …)
• attempts: number of tries it took (including the final one)
• hasAiOverview: whether Google showed an AI Overview by default on the search results page
• answerSource: where the answer came from — ai_overview (shown by default), ai_mode (only after switching to the AI Mode tab, i.e. NO AI summary was shown by default), or null (no AI answer at all)
• responseText: the AI answer as plain text (expanded AI Overview or AI Mode answer), or null
• responseMarkdown: the AI answer with formatting preserved (headings, lists, tables), or null
• sources: array of { position, text, url, domain } cited by the answer (URLs normalized, deduplicated)
• sourceDomains: unique cited domains (aggregated GEO metric)
• responseTimeMs, promptIndex, totalPrompts, timestamp, url (the AI Mode URL when the fallback was used)

Pricing

This Actor uses pay per event + usage pricing:

• One query-processed event per successfully processed prompt. Failed prompts (captcha, crash) are not charged.
• Apify platform usage (residential proxy traffic, compute) is billed separately to you at cost — so you always see exactly what a run consumes.

How it works: real browser (trade-offs)

This Actor drives a real stealth browser (Camoufox/Firefox) through Google and reads the AI Overview exactly as a user sees it — rather than calling a third-party SERP API. That choice has clear pros and cons:

Advantages

• Always live, never cached — every result is fetched fresh from Google in real time. No stale, hours-old cache (a common downside of SERP-API wrappers), which matters because AI Overviews change frequently.
• No third-party dependency — no external API key, no per-call API fee, no reliance on someone else's uptime or terms. You only pay Apify usage.
• Full extraction control — text and markdown, all cited sources with type (citation/source/more), aggregated domains, and the AI Mode fallback when no Overview is shown. An API only returns whatever fields it chooses to.
• Transparent cost — you see the exact compute/proxy usage per run.

Limitations

• Best-effort reliability — Google actively blocks automated search. The Actor uses residential proxies with fresh per-session IP rotation, geo-matched IPs and stealth browsing, but success rates vary by time, region and query; some runs get captcha'd. Blocked prompts are reported cleanly as success: false with an error (and are not charged), so you can re-run them — ideally at a different time or with a different proxyCountry.
• Heavier & slower per query than a plain HTTP API wrapper (it runs a full browser), which is why memory and run time are higher.

If you need guaranteed, high-volume reliability above all else, a SERP-API-based scraper may fit better; if you want live, uncached data with full control and no external dependency, this browser approach is the right tool.

Notes

• AI Overviews are not shown for every query (e.g. navigational queries) and their availability varies by country and language.
• Whether an AI Overview appears can also depend on the IP location of the proxy. By default proxyCountry automatically matches your search country (countryCode / gl) — a matching IP is far less likely to be blocked.