# ChatGPT Scraper & AI Search Brand Visibility Tracker (`scrape-lads/ai-answer-scraper`) Actor

Scrape the answers ChatGPT, Google Gemini and Google AI Overviews give to your prompts, as real logged-out users see them. Markdown answer, cited sources, and brand mention tracking (your brand vs competitors) for GEO / AI search visibility.

- **URL**: https://apify.com/scrape-lads/ai-answer-scraper.md
- **Developed by:** [Scrape Lads](https://apify.com/scrape-lads) (community)
- **Categories:** AI, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $16.00 / 1,000 ai answers

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## ChatGPT Scraper & AI Search Brand Visibility Tracker

**AI Search Answer Scraper** asks ChatGPT, Google Gemini and Google AI Overviews your prompts, exactly as a real logged-out user would, and returns each **AI answer as Markdown**, the **cited sources** (URL, title, domain, position) and a **brand visibility report**: is your brand mentioned, how many times, in what order next to your competitors, and is your website cited.

It is built for **GEO (generative engine optimization)**, **AEO (answer engine optimization)** and **LLM SEO**: track your share of voice in AI search, see which pages the AI engines cite for your category, and monitor competitors over time. No API keys, no ChatGPT account, no login.

### What does the AI Search Answer Scraper do?

- **ChatGPT scraper**: types your prompt into chatgpt.com (logged out, with ChatGPT's own web search) and reads the full answer with every citation.
- **Gemini scraper**: asks gemini.google.com the same way and captures the answer plus the web sources Gemini grounded it on.
- **Google AI Overview scraper**: fetches the normal Google results page and extracts the AI Overview (the AI summary at the top), with its citations. Queries without an AI Overview are reported as such and are free.
- **Brand mention tracker**: for every brand you list (yours and your competitors), each answer row says `mentioned`, `mentionCount`, `rankAmongBrands` (1 = named first), `citedDomain` (your website is a source) and `bestSourcePosition`.

Every row is labelled `source: "ui"`: the data comes from the public consumer interfaces, not from an API model that answers differently. That is what your customers actually see.

### What data can you extract from ChatGPT, Gemini and AI Overviews?

| 🤖 AI engine | ❓ Your prompt | 📝 Full answer in Markdown |
|---|---|---|
| 🏷️ Brand mentioned or not | 🥇 Brand rank in the answer | 📊 Share of voice |
| 📍 Mention position score | 🙂 Brand stance | 🆚 Competitors mentioned |
| 🔗 Your site cited as a source | 📚 Cited sources (URL, title, position) | 🌐 Cited domains |
| 🔢 Citation count | 🕒 Scraped at | ⚠️ Status and errors |

### Why scrape AI answers instead of calling the OpenAI or Gemini API?

The API models (OpenAI Responses with web search, Gemini grounding, Perplexity Sonar) are different products from the chat apps: other system prompts, other search back-ends, other citations. For AI search visibility you want the answer a real person gets on chatgpt.com or google.com, so this Actor scrapes the real interfaces. It is also cheaper than most API setups for this job, because you pay one flat price per answer.

### How to use it

1. Add your **prompts** the way customers ask them ("What is the best CRM for a small business?").
2. Pick the **AI engines**: ChatGPT, Gemini, Google AI Overview.
3. List your **brands**, one per line. Add aliases and the website after `|`: `HubSpot | hubspot.com`, `Zoho CRM | Zoho | zoho.com`.
4. Choose the **country** and **language**. Click **Start**.
5. Open the **Answers and brand visibility**, **Brand mentions** or **Cited sources** view, or export JSON, CSV or Excel.

#### Input example

```json
{
    "queries": ["What are the best CRM tools for small businesses?"],
    "engines": ["chatgpt", "gemini", "google_ai_overview"],
    "brands": ["HubSpot | hubspot.com", "Salesforce | salesforce.com", "Zoho CRM | Zoho | zoho.com"],
    "country": "US",
    "language": "en"
}
```

### Output: AI answer, sources and brand mentions

One row per prompt and engine:

```json
{
    "query": "What are the best CRM tools for small businesses?",
    "engine": "chatgpt",
    "engineName": "ChatGPT",
    "status": "ok",
    "source": "ui",
    "answerMarkdown": "For most small businesses, I'd start by comparing **HubSpot, Zoho CRM, and Pipedrive**. ... ([Spotsaas](https://www.spotsaas.com/blog/...))",
    "answerText": "For most small businesses, I'd start by comparing HubSpot, Zoho CRM, and Pipedrive. ...",
    "sources": [
        { "position": 1, "url": "https://www.spotsaas.com/blog/best-crm-software-2026-reviews-pricing-feature-comparison", "title": "Best CRM Software 2026", "domain": "spotsaas.com", "citedInline": true },
        { "position": 4, "url": "https://www.hubspot.com/products/crm", "title": "Explore HubSpot", "domain": "hubspot.com", "citedInline": true }
    ],
    "sourceCount": 18,
    "citedDomains": ["spotsaas.com", "decisioncircuit.com", "hubspot.com", "zoho.com"],
    "brandMentions": [
        { "brand": "HubSpot", "mentioned": true, "mentionCount": 8, "rankAmongBrands": 1, "firstMentionOffset": 272, "citedDomain": true, "citedSourceCount": 3, "bestSourcePosition": 4 },
        { "brand": "Salesforce", "mentioned": false, "mentionCount": 0, "rankAmongBrands": null, "firstMentionOffset": null, "citedDomain": false, "citedSourceCount": 0, "bestSourcePosition": null }
    ],
    "brandsMentioned": ["HubSpot", "Zoho CRM", "Pipedrive"],
    "model": null,
    "country": "US",
    "language": "en",
    "note": null,
    "error": null,
    "attempts": 1,
    "durationSec": 30.3,
    "scrapedAt": "2026-10-24T10:02:11.000Z"
}
```

| Field | Meaning |
| --- | --- |
| `status` | `ok`, `no_answer` (e.g. Google showed no AI Overview) or `failed` |
| `answerMarkdown` / `answerText` | The full answer, with citations kept as Markdown links / as plain text |
| `sources[]` | Cited or listed sources in the engine's order: `position`, `url` (tracking params removed), `title`, `domain`, `citedInline` |
| `brandMentions[]` | Per brand: `mentioned`, `mentionCount`, `rankAmongBrands`, `citedDomain`, `bestSourcePosition` |
| `brandsMentioned` | Brands named in the answer, in order of first mention |
| `model` | Model shown by the engine when it says (e.g. Gemini "3.5 Flash-Lite") |

### How much does it cost to scrape ChatGPT answers?

Pay per event, no subscription:

- **$0.02 per AI answer** ($20 per 1,000), for every engine.
- A small start fee per run.
- Failed prompts and Google queries without an AI Overview are **free**.

Tracking 50 prompts on 3 engines every week costs about $3 per week. That is more than 10x cheaper than tools charging $0.20 per search.

### Supported AI engines

| Engine | How | Notes |
| --- | --- | --- |
| ChatGPT | Real browser, logged out, chatgpt.com | Uses ChatGPT's own web search when it decides to search, as for any visitor |
| Google Gemini | Real browser, logged out, gemini.google.com | Logged-out Gemini runs the Flash-Lite model with Google Search grounding |
| Google AI Overview | Google results page | Not every query triggers an AI Overview; those rows are `no_answer` and free |

Perplexity, Microsoft Copilot and Google AI Mode now require a signed-in account or block automated visitors, so they are not supported yet.

### Tips for AI visibility tracking

- Ask the same prompts on a **schedule** (daily or weekly) and chart `rankAmongBrands` and `citedDomain` over time. AI answers vary from run to run, so look at trends across several prompts, not one answer.
- Use **several phrasings** of the same intent ("best CRM for startups", "HubSpot alternatives", "cheapest CRM for a 5 person team").
- Check `citedDomains` to find the review sites, listicles and Reddit threads the AI engines trust in your niche; that is where GEO work pays off.
- Run it from different **countries** to compare local answers.

### Run it on a schedule or via API

Create a task with your prompts and brands, then add a schedule in Apify Console (Schedules > Create). Results can go to Google Sheets, Slack, a webhook or any of the Apify integrations. You can also call the Actor from code with the Apify API or the `apify-client` package, or from an AI agent through the Apify MCP server.

### FAQ

**Is it legal to scrape ChatGPT and Gemini answers?** The Actor only reads public answers that any logged-out visitor gets. It does not log in or bypass paywalls. Check the terms of each service for your use case and do not use the data to build a competing model.

**Why do answers differ between runs?** Generative engines are not deterministic, and ChatGPT decides per prompt whether to search the web. Track several prompts over time.

**Official API vs this Actor?** The OpenAI, Gemini and Perplexity APIs answer from different models and search stacks than the consumer apps, need API keys and are billed per token plus per search. This Actor returns what users see, for a flat price per answer.

**What are the alternatives?** Separate ChatGPT, Perplexity or AI Overview scrapers on the Apify Store, or SaaS GEO tools (Profound, Peec AI, Otterly). This Actor combines several engines and brand tracking in one run at a per-answer price.

**Can I track Perplexity or Copilot?** Not yet: both now require a signed-in session for automated visitors. Open an issue if you need them.

### Related actors

- [Video Transcript Scraper](https://apify.com/scrape-lads/video-transcript-scraper): transcripts from TikTok, Instagram, YouTube and X.
- [Reddit Scraper](https://apify.com/scrape-lads/reddit-scraper): posts and comments, useful to see the Reddit threads AI engines cite.
- [Amazon Product Scraper](https://apify.com/scrape-lads/amazon-product-scraper): product data for the brands AI engines recommend.
- [Similarweb Scraper](https://apify.com/scrape-lads/similarweb-scraper): see how much traffic ChatGPT, Claude and Perplexity actually send to a site.
- [Google AI Mode Scraper](https://apify.com/apify/google-ai-mode-scraper): Apify's own scraper for Google AI Mode answers.

# Actor input Schema

## `queries` (type: `array`):

The questions to ask each AI engine, written the way your customers would ask them ("What is the best CRM for a small business?"). One answer row per prompt and engine.

## `engines` (type: `array`):

Which AI answer engines to ask. ChatGPT and Gemini are asked in a real browser as a logged-out visitor; Google AI Overview is the AI summary at the top of a normal Google results page (not every query gets one).

## `brandName` (type: `string`):

The brand whose AI visibility you track, e.g. "HubSpot". Matched case-insensitively on whole words.

## `brandDomain` (type: `string`):

e.g. hubspot.com. Citations on this domain or any subdomain count as your site being cited (isCitedAsSource).

## `brandAliases` (type: `array`):

Other names for your brand: product names, old names, common spellings ("HubSpot CRM", "Hubspot Sales Hub").

## `competitors` (type: `array`):

One competitor per line. Add aliases or the website after a |: "Zoho CRM | Zoho | zoho.com". Used for rank among brands, share of voice and competitor citations.

## `competitorDomains` (type: `array`):

Competitor websites, if you did not add them after a | above. Each domain is paired with the competitor whose name matches it, otherwise in list order.

## `country` (type: `string`):

Where the questions are asked from: proxy exit country and Google's gl parameter. Answers and sources differ by country.

## `language` (type: `string`):

Interface language (Google hl, Gemini hl, browser language), e.g. en, de, fr, es. Write your prompts in the same language.

## `includeSummary` (type: `boolean`):

Adds one row (recordType "summary") with your visibility score 0-100 (40% mention rate, 30% position, 20% share of voice, 10% citation rate), a per-engine breakdown, competitor metrics, the most cited domains and your most cited pages. Free.

## `includeOrganicResults` (type: `boolean`):

Google AI Overview rows also get the organic results and People Also Ask questions from the same results page, at no extra cost.

## `analyzeSentiment` (type: `boolean`):

A small LLM reads each answer and labels every mentioned brand as recommended, neutral or warned against, with 3-5 attributes the answer uses ("free plan", "steep learning curve"). Included in the answer price.

## `fanOutVariants` (type: `integer`):

Generate this many rewordings of each prompt (budget, use case, comparison angles) and ask them too, like the fan-out AI engines do internally. Each variant is a normal answer per engine.

## `maxBudgetUsd` (type: `number`):

Safety cap: the run estimates its cost up front and skips answers that would go over this amount. Leave empty to use the run's normal spending limit.

## `webhookUrl` (type: `string`):

Optional. When the run finishes, a POST with the run id, the dataset URL, the run summary and the report row is sent here (e.g. Zapier, Make, n8n, Slack workflow).

## `demoMode` (type: `boolean`):

Runs only the first prompt on each engine and charges nothing. Use it to check the engines respond before a big run.

## `maxConcurrency` (type: `integer`):

How many ChatGPT / Gemini prompts run at the same time (split between the engines). More is faster but needs more memory: about 1 per GB.

## `maxRetries` (type: `integer`):

When an engine refuses or times out, the prompt is retried on a fresh browser session and IP this many times. Failed rows are free.

## Actor input object example

```json
{
  "queries": [
    "What are the best CRM tools for small businesses?"
  ],
  "engines": [
    "chatgpt",
    "gemini",
    "google_ai_overview"
  ],
  "brandName": "HubSpot",
  "brandDomain": "hubspot.com",
  "competitors": [
    "Salesforce | salesforce.com",
    "Zoho CRM | Zoho | zoho.com",
    "Pipedrive | pipedrive.com"
  ],
  "country": "US",
  "language": "en",
  "includeSummary": true,
  "includeOrganicResults": true,
  "analyzeSentiment": false,
  "fanOutVariants": 0,
  "demoMode": false,
  "maxConcurrency": 2,
  "maxRetries": 2
}
```

# Actor output Schema

## `answers` (type: `string`):

No description

## `fullResults` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "What are the best CRM tools for small businesses?"
    ],
    "brandName": "HubSpot",
    "brandDomain": "hubspot.com",
    "competitors": [
        "Salesforce | salesforce.com",
        "Zoho CRM | Zoho | zoho.com",
        "Pipedrive | pipedrive.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrape-lads/ai-answer-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["What are the best CRM tools for small businesses?"],
    "brandName": "HubSpot",
    "brandDomain": "hubspot.com",
    "competitors": [
        "Salesforce | salesforce.com",
        "Zoho CRM | Zoho | zoho.com",
        "Pipedrive | pipedrive.com",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("scrape-lads/ai-answer-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "What are the best CRM tools for small businesses?"
  ],
  "brandName": "HubSpot",
  "brandDomain": "hubspot.com",
  "competitors": [
    "Salesforce | salesforce.com",
    "Zoho CRM | Zoho | zoho.com",
    "Pipedrive | pipedrive.com"
  ]
}' |
apify call scrape-lads/ai-answer-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrape-lads/ai-answer-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/canM5qqxdVCX96djI/builds/5yHio2h2XoMgxvSa0/openapi.json