Google AI Overview Scraper and Google AI Mode Scraper API

Scrape Google AI Overviews and Google AI Mode answers at scale, without a browser. This google ai overview scraper loads the Google AI Mode search page (udm=50) and calls Google's internal streaming endpoint directly, so you get the full AI-generated answer plus the exact web sources it cited, returned as clean JSON. One row per query. Pay only for the queries that actually return an AI answer.

No official Google API exposes AI Overview content. This google ai mode scraper fills that gap: point it at a list of questions, get back the answer text, the sentence breakdown, and the distinct source domains Google grounded its response on.

What you get

• Full AI answer text, the complete AI Overview / AI Mode response as plain text (answer_text).
• Sentence breakdown, the same answer split into individual sentences (answer_sentences) for citation matching and content analysis.
• Cited source domains, the distinct websites Google linked as sources for the answer (sources), plus a source_count.
• Present flag, a boolean (present) telling you whether an AI answer actually rendered for that query. You are billed only when this is true.
• Per-country and per-language. Set country (gl) and language (hl) to run the same questions across markets.

Why scrape Google AI Overviews

AI Overviews are reshaping how search visibility works. When Google answers a query inline with an AI-generated summary, it cites a handful of source websites and pushes the classic ten blue links down the page. If your brand is one of those cited sources, you capture attention at the very top. If it is not, a competitor probably is, and the click you used to win from organic ranking never happens.

That shift matters for anyone who cares about search. Traditional rank tracking tells you where you sit in the organic list. It says nothing about whether Google's AI is quoting you, a competitor, or nobody. Those are different questions, and the AI answer is increasingly the one users read first.

There is no official API for this data. Google does not publish AI Overview content or its cited sources through any documented endpoint. The only way to know whether you are being cited is to run the query the way a real user would and read the result. This actor does that programmatically, across as many questions and markets as you need, so you can track AI citation the same way you already track keyword rankings.

Input

Field	Type	Default	Description
queries	array	required	Questions or search queries to run through Google AI Mode. Question-style queries work best (for example "is the lego 10313 worth buying", "what is web scraping used for"). Product names, models, and EANs also ground well and return merchant sources.
country	string	us	Two-letter country code (gl): us, nl, de, fr, es, it, uk.
language	string	en	Two-letter language code (hl).
retries	integer	3	Cookie-rotation retries (0-6). Google renders AI Mode tokens only for a region-matched session; on a miss the actor rotates to a fresh session and retries. 3 is plenty.

Output

Every run pushes one record per query to the Apify dataset. Here is an annotated example:

{
  "query": "what is web scraping used for",
  "present": true,
  "answer_text": "Web scraping is used to automatically collect data from websites for purposes such as price monitoring, market research, lead generation, and training machine learning models...",
  "answer_sentences": [
    "Web scraping is used to automatically collect data from websites.",
    "Common purposes include price monitoring, market research, lead generation, and training machine learning models."
  ],
  "sources": ["wikipedia.org", "zapier.com", "apify.com"],
  "source_count": 3,
  "country": "us",
  "language": "en",
  "attempts": 1,
  "raw_len": 48213,
  "error": null,
  "fetched_in_seconds": 4.7
}

• answer_text is the full AI answer. answer_sentences is the same content split for easier matching against your own copy or a competitor's.
• sources is the deduplicated list of domains Google cited. This is the field most SEO teams care about, it tells you who got quoted.
• present: false means no AI answer rendered for that query. The row still comes back (with an error reason), and you are not billed for it.
• attempts shows how many cookie sessions were tried before the answer rendered, raw_len is the size of the raw response, and fetched_in_seconds is the wall-clock time for that query.

Download the dataset as JSON, JSONL, CSV, or Excel, stream it via the Apify API, or pipe it to webhooks, S3, BigQuery, or any of Apify's integrations.

Use cases

SEO teams tracking AI citation. Feed in the questions your target keywords answer, run the actor on a schedule, and watch the sources field over time. You see exactly which queries cite your domain, which cite competitors, and which ignore you. That is the AI-era equivalent of a rank-tracking report, and data you cannot get from a standard SERP tool.

Brand monitoring. Run branded and category questions ("is BRAND worth it", "best CRM for small business") across your key markets. If Google's AI mentions you, your reputation content is landing. If it quotes a review site or a competitor instead, you have found a gap to close.

Content gap analysis. Compare the answer_sentences Google generates against your existing pages. Where the AI answer covers a point your content misses, you have a content brief. Where a competitor domain shows up in sources repeatedly, you know which pages to study and beat.

GEO and AEO optimization. Generative Engine Optimization and Answer Engine Optimization are about getting cited by AI answers, not just ranking in blue links. This actor gives you the measurement layer: run the same question set before and after a content change and see whether your citation rate moved. Without that feedback loop, GEO work is guesswork.

How it compares

	This actor (s-r/google-ai-overview)	clearpath/google-ai-overview
Per-query rate	$0.00999 (billed only when present)	$0.01499 (billed per query)
Per 1,000 queries	$9.99	$14.99
Actor-start fee	$0.002 per run	$0.09 per run
Engine	Direct pure-httpx (no browser)	Browser-based
Billing basis	Only queries that returned an answer	Per query attempted

Two things stand out. We undercut the per-query rate by 20 percent. And our actor-start fee is $0.002 against their $0.09, which is 45 times cheaper on the fee you pay every single run. On a typical small run, clearpath's $0.09 start fee alone can be a third of the bill. Our direct httpx engine (no browser) also has near-zero marginal cost, which is what lets us hold the lower price.

Because the present flag gates billing, you are never charged for a query that returned no AI answer, unlike a per-query model.

Pricing

Pay-per-event, billed through Apify:

• $0.00999 per overview, charged only when present is true (an AI answer actually rendered). Queries that return no answer cost nothing.
• $0.002 actor-start fee, a flat fee per run.

That works out to $9.99 per 1,000 answered queries plus $0.002 per run. No monthly subscription, no minimum spend. You only pay for the AI answers you actually receive.

Limits and gotchas

• AI answers render only for some query types. Google does not attach an AI Overview to every search. Informational and comparison questions trigger them far more often than navigational or transactional ones.
• Question-style queries work best. Phrase inputs as real questions ("is X worth buying", "what is Y used for"). Product names, models, and EANs also ground well and tend to return merchant sources.
• present: false is not a failure and is not billed. It simply means Google served no AI answer for that query. The row still returns with an error reason so you know why.
• Cookie-rotation retries handle region mismatch. Google renders AI Mode tokens only for a session that matches the requested region. On a miss the actor rotates to a fresh session and retries, up to the retries value you set.
• Free-tier cap. Free Apify accounts are capped at 10 results per run. Upgrade to a paid Apify plan to remove the cap.
• Results reflect live Google. There is no cache or middleware. Answers can vary between runs as Google updates its models and grounding, which is exactly why scheduled monitoring beats one-off checks.

FAQ

How do I track if my brand appears in Google AI Overviews?

Build a list of the questions your customers ask ("best X for Y", "is BRAND worth it", "X vs competitor") and run them through this actor on a schedule. Watch the sources field. When your domain appears there, Google's AI is citing you. Because the actor returns distinct cited domains for every query, you can measure your citation rate across dozens of questions and watch it change as you publish content.

Can I scrape Google AI Mode answers?

Yes, that is exactly what this actor does. It loads the Google AI Mode page (udm=50) and reads the streamed answer directly, returning the full text, a sentence breakdown, and the cited sources. You do not need a browser, a proxy setup, or any anti-bot maintenance. The actor handles session and region matching for you.

Which queries trigger an AI Overview?

Informational and comparison questions trigger them most reliably. Think "what is X used for", "how does Y work", "is Z worth buying", and product-comparison phrasing. Purely navigational queries (a brand name on its own) and highly transactional ones often do not get an AI answer. If a query returns present: false, Google chose not to render one, and you are not charged for it.

Why did some of my queries return no answer?

A present: false result means Google served no AI Overview for that query in that market. That is normal, not every search gets one. It also happens when the query type is not a good fit (too navigational). Rephrasing as a question usually helps. Either way, unanswered queries are free.

Can I run this across multiple countries?

Yes. Set country (gl) and language (hl) to the market you want, and run the same question set per market. Google's AI answers and cited sources differ by region and language, so running the same questions across us, nl, de, fr, es, it, and uk shows you where your citation footprint is strong and where it is thin.

Related Actors

• Google Lens - Reverse image search: find where an image appears online plus matching products.
• Google News - Scrape Google News results for any query.
• Bing Serp - Scrape Bing search engine results pages.