# AI Search Scraper – Google AI Overview Answers & Citations (`inovaflow/ai-search-scraper`) Actor

Scrape the real Google AI Overview answer people see in search results — the answer text, its cited sources with resolved URLs, and the page-1 organic results — for any query, per country and language. Present / no-answer flag per row, no API keys, no login. Dataset-only, MCP-ready.

- **URL**: https://apify.com/inovaflow/ai-search-scraper.md
- **Developed by:** [inovaflow](https://apify.com/inovaflow) (community)
- **Categories:** SEO tools, AI, Agents
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 ai answers

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**Get the real Google AI Overview answer — the one people actually see at the top of Google search results — for any query, in any country, with every source it cites.** Give it your queries and markets; each row carries the full answer text, the cited pages with resolved URLs, the page-1 organic results of the same search, and a clear flag telling you whether Google shows an AI Overview for that query at all. No API keys, no login, no browser.

AI answers are where your customers now look first, and they are not the same as what a chat API returns: the search answer is grounded in Google's index for that country and cites specific pages. This scraper reads that answer from Google's own result page, so what you get is what a searcher in that market gets.

### AI search scraper: what you get

One row per query × country:

| Field | What it tells you |
| --- | --- |
| `present`, `status` | `present: true` — an AI Overview was shown and captured; `present: false` — Google shows no AI Overview for this query (a real, useful result); `status` is `ok`, `generating` (Google is still generating the answer for this query and market — re-run later), `blocked` or `error` |
| `answerText`, `answerCharCount` | The complete answer, paragraphs and list items on separate lines, without UI text or source chips |
| `citations[]`, `citationCount` | The cited pages in the order Google shows them — `url` (resolved), `domain`, `title`, `position`, `kind` (`inline` source chip in the answer, or `related` link under it) |
| `organicResults[]`, `organicCount` | The page-1 organic results of the same search, in order — `position`, `url` (resolved), `domain`, `title`, `displayUrl` — so you see what ranks next to what the AI says (ads and "People also ask" excluded; switch off with *Include page-1 organic results*) |
| `query`, `country`, `countryCode`, `language`, `searchDomain`, `sourceUrl` | What was asked, where, and the exact search page it came from |
| `unavailableReason` | Why there is no answer: `no_aio_module` (Google shows none), `aio_not_available_msg`, `aio_generating`, `google_blocked`, `transport_error` |
| `searchQuery`, `aiOverview` | The same objects the first-party Google Search Scraper emits, so existing pipelines built on it can switch sources without changing their parsers |
| `metadata` | Diagnostics: the localized heading, whether Google marked the overview as an AI Mode answer, citation counts by kind |

Three dataset views: **AI answers** (one line per query and country), **Citations** (the sources per answer) and **Organic results** (the page-1 ranking next to the answer).

### How it works

1. **Search** — each query is run on the Google domain of the chosen country (`google.de` for Germany, `google.co.uk` for the UK) with the country's interface language, through a search-engine-grade proxy pool, exactly as a desktop visitor would.
2. **Read the answer** — the AI Overview block is located structurally (not by fragile class names), the answer text is extracted with its paragraph and list structure, and UI strings and source chips are stripped out.
3. **Resolve the sources** — every cited page is a redirect token on the page; each unique token is resolved to the real URL once per run, so `citations[].url` is the page itself.
4. **Deliver honestly** — a query without an AI Overview is a `present: false` row, not an error; a page Google is still generating is `generating`; a block page is `blocked`. Rows without an answer are never charged.

No API keys, no login, no cookies, no nested scrapers.

### Who uses it

- **SEO and AEO teams** — see the exact answer, the cited pages and the page-1 organic results for your money queries, per market, every day: is the AI citing you, and do you rank underneath it?
- **Brand and competitor monitoring** — feed the rows into a mention counter or a share-of-voice sheet; the `aiOverview` object is a drop-in for pipelines built on the first-party Google Search Scraper.
- **Agents and automations** — a keyword-discoverable, MCP-callable step that turns a list of questions into structured AI answers with citations, unattended.

### Set it up in a minute

1. Paste your **Queries** (one per line).
2. Pick the **Countries** (default: United States). Optionally set the **Interface language**.
3. Start. Rows arrive as each query is finished; the run summary (presence rate per country, citation stats) is in the `OUTPUT` record.

Citation and retry settings are collapsed below the essentials; the defaults are tuned for unattended runs.

### Use it from an agent or the API

```json
{ "queries": ["best crm for small business", "what is model context protocol"], "countries": ["us", "uk"] }
```

```json
{ "queries": ["bestes CRM für kleine Unternehmen"], "countries": ["de"], "language": "de", "maxCitations": 10, "includeAnswerHtml": true }
```

Agents may also pass `prompts` or `keywords` instead of `queries`, `country` as a comma-separated string, and query objects `{ query }`. Results are in the default dataset (`?view=answers`, `?view=citations`); the run summary is in the `OUTPUT` record of the run's key-value store. Through the Apify MCP server, call `inovaflow/ai-search-scraper` with the same input.

### Output example

```json
{
  "query": "best crm for small business",
  "surface": "google_ai_overview",
  "country": "us",
  "countryCode": "US",
  "language": "en",
  "searchDomain": "google.com",
  "status": "ok",
  "present": true,
  "unavailableReason": null,
  "answerText": "HubSpot is the best overall CRM for small businesses because it offers a powerful free tier and scales smoothly as your company grows. A CRM (Customer Relationship Management) tool is software that helps you track leads, manage customer contacts, and organize sales.\nTop Small Business CRM Options\nHubSpot: Best all-in-one option for marketing-driven teams. It features a popular free tier (up to two users), email templates, and contact management.\nPipedrive: Best for straightforward, visual sales pipelines. Plans start around $12.50 to $15 per user, per month.",
  "answerCharCount": 690,
  "citations": [
    { "url": "https://www.uschamber.com/co/start/strategy/low-cost-crm-tools", "domain": "uschamber.com", "title": "U.S. Chamber of Commerce", "position": 1, "kind": "inline", "resolved": true },
    { "url": "https://www.pcmag.com/picks/the-best-small-business-crm-software", "domain": "pcmag.com", "title": "PCMag", "position": 3, "kind": "inline", "resolved": true },
    { "url": "https://www.reddit.com/r/CRM/comments/…", "domain": "reddit.com", "title": "Best and inexpensive CRM for small business - Reddit", "position": 4, "kind": "related", "resolved": true }
  ],
  "citationCount": 5,
  "organicResults": [
    { "position": 1, "url": "https://slack.com/blog/best-crm-for-small-business", "domain": "slack.com", "title": "Best CRM for Small Business in 2026", "displayUrl": "https://slack.com/blog/best-crm-for-small-busi...", "resolved": true },
    { "position": 2, "url": "https://www.zoho.com/crm/small-business-crm-software/", "domain": "zoho.com", "title": "The Best CRM Software for Small Businesses", "displayUrl": "https://www.zoho.com/crm/small-business-crm-so...", "resolved": true }
  ],
  "organicCount": 9,
  "sourceUrl": "https://www.google.com/search?q=best+crm+for+small+business&hl=en&gl=us",
  "aiOverview": { "type": "live", "content": "HubSpot is the best overall CRM …", "sources": [{ "url": "https://www.uschamber.com/co/start/strategy/low-cost-crm-tools", "title": "U.S. Chamber of Commerce", "description": "" }] }
}
```

### Pricing

Pay per event: **$0.01 per AI answer** delivered (a query × country with the answer text and its citations), plus a small per-run start fee. Queries where Google shows no AI Overview, is still generating one, or served a block page are free — and so are the organic results, which come from the same page.

### Good to know

- **No AI Overview is a result, not a failure.** Google shows an AI Overview for roughly three quarters of informational and commercial English queries; the rest come back as `present: false` so you can track when an answer appears.
- **Live-generated answers.** For some queries — mostly non-English ones in their own market — Google generates the overview live in the browser instead of serving it with the page. Those rows are `status: generating` (after two automatic re-fetches); re-run later, or ask the question in English, which is served with the page on every market's Google domain tested.
- **What is visible is what you get.** The answer text is the overview as rendered on the result page; a few very long overviews are shown by Google with a trailing section collapsed, and the collapsed part is not on the page.
- **Organic results are free.** They are read from the same result page as the answer, so a query with no AI Overview still gives you its page-1 ranking — at no charge.
- **Countries** — 49 markets, each on its own Google domain with its main language by default; override with *Interface language*.
- Results reflect what Google shows at the moment of the run; `sourceUrl` and `metadata` carry the evidence for every row.

# Actor input Schema

## `queries` (type: `array`):

Search queries, one per line — the questions your customers type into Google (`best CRM for small business`). Each query is checked on every selected country. Also accepted as `prompts` or `keywords`, or as objects {query}.

## `countries` (type: `array`):

Markets to check — each one is a separate Google domain and result page (a query × country pair is one row). Default: United States.

## `language` (type: `string`):

Google interface language (`hl`), e.g. `en`, `de`, `cs`. Leave empty to use each country's main language. Tip: AI Overviews are served most consistently when the query language matches this.

## `surfaces` (type: `array`):

Which AI answers to return. Google AI Overview is the answer box at the top of Google search results.

## `maxCitations` (type: `integer`):

Cap on cited sources returned per answer (inline source chips first, then the related links under the answer). 0 = answer text only.

## `resolveCitationUrls` (type: `boolean`):

Google links every source through a redirect token; when on, each token is resolved to the real page URL (one small request per unique source, cached per run).

## `includeAnswerHtml` (type: `boolean`):

Also store the raw HTML of the AI Overview block in `answerHtml` (large; useful for audits and custom parsing).

## `includeOrganicResults` (type: `boolean`):

Also return the page-1 organic results of the same search in `organicResults` (position, resolved URL, domain, title) — what ranks next to what the AI answers. Read from the same page, so it adds no search request and no charge.

## `generationRetries` (type: `integer`):

When Google shows the AI Overview as still being generated (a one-off on common queries, the norm for niche or non-English ones), re-fetch the page this many times a few seconds apart before reporting `generating`. Each re-fetch is one more search request.

## `maxRetries` (type: `integer`):

Extra attempts when Google serves a block page or the request fails.

## `maxConcurrency` (type: `integer`):

How many query × country pairs are fetched at once.

## `maxItems` (type: `integer`):

Upper bound on query × country × surface rows produced in one run.

## Actor input object example

```json
{
  "queries": [
    "best running shoes for flat feet",
    "is coffee good for you"
  ],
  "countries": [
    "us"
  ],
  "language": "en",
  "surfaces": [
    "google_ai_overview"
  ],
  "maxCitations": 20,
  "resolveCitationUrls": true,
  "includeAnswerHtml": false,
  "includeOrganicResults": true,
  "generationRetries": 2,
  "maxRetries": 2,
  "maxConcurrency": 5,
  "maxItems": 1000
}
```

# Actor output Schema

## `answers` (type: `string`):

One row per query and country: status, present flag, answer text, citation count.

## `citations` (type: `string`):

The cited sources per answer (URL, domain, title, position).

## `summary` (type: `string`):

Counts by status and country.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "best crm for small business",
        "what is model context protocol"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("inovaflow/ai-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "queries": [
        "best crm for small business",
        "what is model context protocol",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("inovaflow/ai-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "best crm for small business",
    "what is model context protocol"
  ]
}' |
apify call inovaflow/ai-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,inovaflow/ai-search-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OGyMApUHlmirQknSZ/builds/LvWz2bKtCX4m4Z6Sg/openapi.json