# Google AI Overview & AI Mode Scraper (`s-r/google-ai-overview`) Actor - **URL**: https://apify.com/s-r/google-ai-overview.md - **Developed by:** [SR](https://apify.com/s-r) (community) - **Categories:** AI, Marketing - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Google AI Overview Scraper and Google AI Mode Scraper API Scrape Google AI Overviews and Google AI Mode answers at scale, without a browser. This google ai overview scraper loads the Google AI Mode search page (udm=50) and calls Google's internal streaming endpoint directly, so you get the full AI-generated answer plus the exact web sources it cited, returned as clean JSON. One row per query. Pay only for the queries that actually return an AI answer. No official Google API exposes AI Overview content. This google ai mode scraper fills that gap: point it at a list of questions, get back the answer text, the sentence breakdown, and the distinct source domains Google grounded its response on. ### What you get - **Full AI answer text**, the complete AI Overview / AI Mode response as plain text (`answer_text`). - **Sentence breakdown**, the same answer split into individual sentences (`answer_sentences`) for citation matching and content analysis. - **Cited source domains**, the distinct websites Google linked as sources for the answer (`sources`), plus a `source_count`. - **Present flag**, a boolean (`present`) telling you whether an AI answer actually rendered for that query. You are billed only when this is true. - **Per-country and per-language**. Set `country` (gl) and `language` (hl) to run the same questions across markets. ### Why scrape Google AI Overviews AI Overviews are reshaping how search visibility works. When Google answers a query inline with an AI-generated summary, it cites a handful of source websites and pushes the classic ten blue links down the page. If your brand is one of those cited sources, you capture attention at the very top. If it is not, a competitor probably is, and the click you used to win from organic ranking never happens. That shift matters for anyone who cares about search. Traditional rank tracking tells you where you sit in the organic list. It says nothing about whether Google's AI is quoting you, a competitor, or nobody. Those are different questions, and the AI answer is increasingly the one users read first. There is no official API for this data. Google does not publish AI Overview content or its cited sources through any documented endpoint. The only way to know whether you are being cited is to run the query the way a real user would and read the result. This actor does that programmatically, across as many questions and markets as you need, so you can track AI citation the same way you already track keyword rankings. ### Input | Field | Type | Default | Description | |---|---|---|---| | `queries` | array | **required** | Questions or search queries to run through Google AI Mode. Question-style queries work best (for example "is the lego 10313 worth buying", "what is web scraping used for"). Product names, models, and EANs also ground well and return merchant sources. | | `country` | string | `us` | Two-letter country code (gl): us, nl, de, fr, es, it, uk. | | `language` | string | `en` | Two-letter language code (hl). | | `retries` | integer | `3` | Cookie-rotation retries (0-6). Google renders AI Mode tokens only for a region-matched session; on a miss the actor rotates to a fresh session and retries. 3 is plenty. | ### Output Every run pushes one record per query to the Apify dataset. Here is an annotated example: ```json { "query": "what is web scraping used for", "present": true, "answer_text": "Web scraping is used to automatically collect data from websites for purposes such as price monitoring, market research, lead generation, and training machine learning models...", "answer_sentences": [ "Web scraping is used to automatically collect data from websites.", "Common purposes include price monitoring, market research, lead generation, and training machine learning models." ], "sources": ["wikipedia.org", "zapier.com", "apify.com"], "source_count": 3, "country": "us", "language": "en", "attempts": 1, "raw_len": 48213, "error": null, "fetched_in_seconds": 4.7 } ```` - `answer_text` is the full AI answer. `answer_sentences` is the same content split for easier matching against your own copy or a competitor's. - `sources` is the deduplicated list of domains Google cited. This is the field most SEO teams care about, it tells you who got quoted. - `present: false` means no AI answer rendered for that query. The row still comes back (with an `error` reason), and you are not billed for it. - `attempts` shows how many cookie sessions were tried before the answer rendered, `raw_len` is the size of the raw response, and `fetched_in_seconds` is the wall-clock time for that query. Download the dataset as JSON, JSONL, CSV, or Excel, stream it via the Apify API, or pipe it to webhooks, S3, BigQuery, or any of Apify's integrations. ### Use cases **SEO teams tracking AI citation.** Feed in the questions your target keywords answer, run the actor on a schedule, and watch the `sources` field over time. You see exactly which queries cite your domain, which cite competitors, and which ignore you. That is the AI-era equivalent of a rank-tracking report, and data you cannot get from a standard SERP tool. **Brand monitoring.** Run branded and category questions ("is BRAND worth it", "best CRM for small business") across your key markets. If Google's AI mentions you, your reputation content is landing. If it quotes a review site or a competitor instead, you have found a gap to close. **Content gap analysis.** Compare the `answer_sentences` Google generates against your existing pages. Where the AI answer covers a point your content misses, you have a content brief. Where a competitor domain shows up in `sources` repeatedly, you know which pages to study and beat. **GEO and AEO optimization.** Generative Engine Optimization and Answer Engine Optimization are about getting cited by AI answers, not just ranking in blue links. This actor gives you the measurement layer: run the same question set before and after a content change and see whether your citation rate moved. Without that feedback loop, GEO work is guesswork. ### How it compares | | This actor (s-r/google-ai-overview) | clearpath/google-ai-overview | |---|---|---| | Per-query rate | **$0.00999** (billed only when present) | $0.01499 (billed per query) | | Per 1,000 queries | **$9.99** | $14.99 | | Actor-start fee | **$0.002 per run** | $0.09 per run | | Engine | Direct pure-httpx (no browser) | Browser-based | | Billing basis | Only queries that returned an answer | Per query attempted | Two things stand out. We undercut the per-query rate by 20 percent. And our actor-start fee is $0.002 against their $0.09, which is 45 times cheaper on the fee you pay every single run. On a typical small run, clearpath's $0.09 start fee alone can be a third of the bill. Our direct httpx engine (no browser) also has near-zero marginal cost, which is what lets us hold the lower price. Because the `present` flag gates billing, you are never charged for a query that returned no AI answer, unlike a per-query model. ### Pricing Pay-per-event, billed through Apify: - **$0.00999 per overview**, charged only when `present` is true (an AI answer actually rendered). Queries that return no answer cost nothing. - **$0.002 actor-start fee**, a flat fee per run. That works out to $9.99 per 1,000 answered queries plus $0.002 per run. No monthly subscription, no minimum spend. You only pay for the AI answers you actually receive. ### Limits and gotchas - **AI answers render only for some query types.** Google does not attach an AI Overview to every search. Informational and comparison questions trigger them far more often than navigational or transactional ones. - **Question-style queries work best.** Phrase inputs as real questions ("is X worth buying", "what is Y used for"). Product names, models, and EANs also ground well and tend to return merchant sources. - **`present: false` is not a failure and is not billed.** It simply means Google served no AI answer for that query. The row still returns with an `error` reason so you know why. - **Cookie-rotation retries handle region mismatch.** Google renders AI Mode tokens only for a session that matches the requested region. On a miss the actor rotates to a fresh session and retries, up to the `retries` value you set. - **Free-tier cap.** Free Apify accounts are capped at 10 results per run. Upgrade to a paid Apify plan to remove the cap. - **Results reflect live Google.** There is no cache or middleware. Answers can vary between runs as Google updates its models and grounding, which is exactly why scheduled monitoring beats one-off checks. ### FAQ #### How do I track if my brand appears in Google AI Overviews? Build a list of the questions your customers ask ("best X for Y", "is BRAND worth it", "X vs competitor") and run them through this actor on a schedule. Watch the `sources` field. When your domain appears there, Google's AI is citing you. Because the actor returns distinct cited domains for every query, you can measure your citation rate across dozens of questions and watch it change as you publish content. #### Can I scrape Google AI Mode answers? Yes, that is exactly what this actor does. It loads the Google AI Mode page (udm=50) and reads the streamed answer directly, returning the full text, a sentence breakdown, and the cited sources. You do not need a browser, a proxy setup, or any anti-bot maintenance. The actor handles session and region matching for you. #### Which queries trigger an AI Overview? Informational and comparison questions trigger them most reliably. Think "what is X used for", "how does Y work", "is Z worth buying", and product-comparison phrasing. Purely navigational queries (a brand name on its own) and highly transactional ones often do not get an AI answer. If a query returns `present: false`, Google chose not to render one, and you are not charged for it. #### Why did some of my queries return no answer? A `present: false` result means Google served no AI Overview for that query in that market. That is normal, not every search gets one. It also happens when the query type is not a good fit (too navigational). Rephrasing as a question usually helps. Either way, unanswered queries are free. #### Can I run this across multiple countries? Yes. Set `country` (gl) and `language` (hl) to the market you want, and run the same question set per market. Google's AI answers and cited sources differ by region and language, so running the same questions across us, nl, de, fr, es, it, and uk shows you where your citation footprint is strong and where it is thin. ### Related Actors - [Google Lens](https://apify.com/s-r/google-lens) - Reverse image search: find where an image appears online plus matching products. - [Google News](https://apify.com/s-r/google-news) - Scrape Google News results for any query. - [Bing Serp](https://apify.com/s-r/bing-serp) - Scrape Bing search engine results pages. # Actor input Schema ## `queries` (type: `array`): Questions or topics to run through Google AI Mode. Question-style queries work best ('is the lego 10313 worth buying', 'what is web scraping used for'). Product names, models and EANs also ground well and return merchant sources. ## `country` (type: `string`): Two-letter country code (gl) — us, nl, de, fr, es, it, uk. ## `language` (type: `string`): Two-letter language code (hl). ## `retries` (type: `integer`): Google only renders AI Mode tokens for a region-matched session. On a miss we rotate to a fresh session and retry. 3 is plenty. ## Actor input object example ```json { "queries": [ "what is web scraping used for" ], "country": "us", "language": "en", "retries": 3 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "queries": [ "what is web scraping used for" ], "country": "us", "language": "en", "retries": 3 }; // Run the Actor and wait for it to finish const run = await client.actor("s-r/google-ai-overview").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "queries": ["what is web scraping used for"], "country": "us", "language": "en", "retries": 3, } # Run the Actor and wait for it to finish run = client.actor("s-r/google-ai-overview").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "queries": [ "what is web scraping used for" ], "country": "us", "language": "en", "retries": 3 }' | apify call s-r/google-ai-overview --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=s-r/google-ai-overview", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Google AI Overview & AI Mode Scraper", "description": "", "version": "0.1", "x-build-id": "dtXOtsQJMGP7hdiIg" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/s-r~google-ai-overview/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-s-r-google-ai-overview", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/s-r~google-ai-overview/runs": { "post": { "operationId": "runs-sync-s-r-google-ai-overview", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/s-r~google-ai-overview/run-sync": { "post": { "operationId": "run-sync-s-r-google-ai-overview", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "queries" ], "properties": { "queries": { "title": "Questions / search queries", "type": "array", "description": "Questions or topics to run through Google AI Mode. Question-style queries work best ('is the lego 10313 worth buying', 'what is web scraping used for'). Product names, models and EANs also ground well and return merchant sources.", "items": { "type": "string" } }, "country": { "title": "Country", "type": "string", "description": "Two-letter country code (gl) — us, nl, de, fr, es, it, uk.", "default": "us" }, "language": { "title": "Language", "type": "string", "description": "Two-letter language code (hl).", "default": "en" }, "retries": { "title": "Cookie-rotation retries", "minimum": 0, "maximum": 6, "type": "integer", "description": "Google only renders AI Mode tokens for a region-matched session. On a miss we rotate to a fresh session and retry. 3 is plenty.", "default": 3 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```