# ChatGPT, Claude, Gemini & Perplexity AI Search Scraper (`stefanie-rink/ai-search-scraper`) Actor

Scrape ChatGPT, Claude, Gemini and Perplexity AI search answers with cited sources. A paid Apify plan is required. No AI API keys needed.

- **URL**: https://apify.com/stefanie-rink/ai-search-scraper.md
- **Developed by:** [Steafanie Braid](https://apify.com/stefanie-rink) (community)
- **Categories:** SEO tools, AI, Automation
- **Stats:** 10 total users, 9 monthly users, 93.6% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $33.33 / 1,000 engine answer in a multi-engine searches

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## ChatGPT, Claude, Gemini & Perplexity AI Search Scraper

**A paid Apify plan is required.** Free-plan runs stop before AI requests. Upgrade your Apify plan to use this Actor.

**AI Search Scraper** sends your questions to **ChatGPT Search**, **Claude Haiku**, **Google Gemini**, and **Perplexity Sonar Pro** in one run.
ChatGPT and Claude use native web search. Gemini uses native Google Search grounding.
Claude uses **Haiku 5.5 with low reasoning effort**.
It returns each engine's **AI answer**, **cited sources and URLs**, **search queries**, and the **source overlap** between engines in a single dataset.
Use it for **GEO (generative engine optimization)**, **AEO (answer engine optimization)**, **brand visibility tracking in AI search**, and citation research.

- ✅ No API keys needed. The Actor includes the connections to all four engines.
- ✅ Questions in any language, with optional location, recency, and domain preferences.
- ✅ Export to JSON, CSV, Excel, or HTML, or connect the results to your tools through the Apify API.

### What does AI Search Scraper do?

AI Search Scraper combines a **ChatGPT scraper**, **Claude scraper**, **Gemini scraper**, and **Perplexity scraper**. For each question, it:

- 🔍 Asks ChatGPT, Claude, Gemini, and Perplexity in parallel, with **web search** turned on.
- 📝 Extracts the full **AI-generated answer** text.
- 🔗 Collects the **citations** with URL, domain, title, and snippet.
- 🧭 Records the **search queries** that the engine sent to the web, when the engine reports them.
- ❓ Keeps **related questions** when the engine suggests them.
- 📊 Compares the engines and shows the **sources they all cite** and the sources that only one engine cites.
- 🌐 Converts Google grounding redirect links into the **actual source URLs**.

### Why scrape AI search engines?

More people now get answers from ChatGPT, Claude, Gemini, and Perplexity instead of a list of blue links.
If an AI answer does not mention your brand or cite your site, those people may never see you.
AI Search Scraper shows you what each AI search engine says and which websites it trusts.

#### Use cases

- **Brand visibility in AI search:** See whether ChatGPT, Claude, Gemini, and Perplexity mention your brand, product, or competitors.
- **GEO and AEO monitoring:** Track which pages AI engines cite, and measure your progress after you change content.
- **Competitor research:** Find the domains and pages that AI answers cite for your market.
- **Content and SEO strategy:** Read the search queries that AI engines generate, and find content gaps.
- **Digital PR and link building:** Find the publications that AI engines cite most, and target them.
- **AI answer comparison and research:** Compare how four AI search engines answer the same question, in any language and location.
- **Scheduled monitoring:** Run the same questions every day or week with Apify schedules, and track changes over time.

### What data can you extract from ChatGPT, Claude, Gemini, and Perplexity?

| Data | Description |
| --- | --- |
| AI answer | The full answer text from each engine. |
| Citations | URL, canonical URL, domain, title, snippet, and position of each cited source. |
| Search queries | The web search queries that the engine used, when the engine reports them. |
| Related questions | Follow-up questions, when the engine returns them. |
| All consulted sources | Optional. Sources that the engine read but did not cite. |
| Source overlap | Sources that two or more engines cite, unique sources per engine, and the overlap rate. |
| Model and usage | Model, token usage, latency, and attempts. |
| Status and warnings | Success, error, timeout, or skipped status for each engine, with a clear reason. |
| Gemini extras | Grounding supports that link answer text to sources, and Google Search suggestions. |

### How to scrape ChatGPT, Claude, Gemini, and Perplexity answers

1. Open the Actor in Apify Console with a paid Apify plan.
2. Enter one or more questions in **Search queries**. For example: `best CRM for small businesses`.
3. Select ChatGPT, Claude, Gemini, Perplexity, or any combination. The default selects all four.
4. Optional: change the default New York City location, source recency, or allowed or blocked domains.
5. Click **Start**.
6. Download the results as JSON, CSV, Excel, or HTML, or read them through the API.

Each question produces one dataset row that contains the results from all selected engines.

### How much does it cost to scrape AI search answers?

AI Search Scraper uses **pay-per-event** pricing. You pay for each successful engine answer.
You do not need your own OpenAI, Anthropic, Google, or Perplexity account.

| What you run | Price |
| --- | --- |
| One question on one engine | $0.05 |
| One question on all four engines | about $0.13 |
| 100 questions on all four engines | about $13.33 |
| Failed or skipped engine results | Free |

With several engines, each successful engine result costs $0.0333.
If only two of four engines succeed, you pay for two.

### Input example

```json
{
  "queries": [
    "best project management tools for small teams",
    "is Notion or Asana better for startups"
  ],
  "engines": ["chatgpt", "gemini", "perplexity", "claude"],
  "perplexityModel": "sonar-pro",
  "userLocation": { "country": "US" },
  "includeAllSources": true
}
```

#### Input options

| Option | Default | Description |
| --- | --- | --- |
| `queries` | — | The questions to ask. `maxQueriesPerRun` sets how many run. |
| `engines` | All four | `chatgpt`, `gemini`, `perplexity`, and `claude`. Select one engine or any combination. |
| `perplexityModel` | **Sonar Pro** | `sonar`, `sonar-pro`, or `sonar-reasoning-pro`. |
| `userLocation` | New York City, US | Two-letter `country` code, `city`, `region`, and IANA `timezone`. Use `{}` for no location preference. |
| `searchRecency` | `any` | `hour`, `day`, `week`, `month`, or `any`. |
| `domainFilter` | None | `allowed` or `blocked` domains, not both. Up to 20 domains with Perplexity, or 100 without it. |
| `searchContextSize` | `medium` | `low`, `medium`, or `high` search depth for ChatGPT and Perplexity. |
| `includeAllSources` | `false` | Include sources that the engine read but did not cite. |
| `includeRelatedQuestions` | `true` | Keep the related questions. |
| `maxCitationsPerEngine` | `20` | Maximum citations per engine. |
| `maxQueriesPerRun` | `50` | Maximum questions per run. Limit: 500. |
| `outputFormat` | `json` | `csv` also saves `RESULTS_CSV` in the key-value store, with one row per question and engine. |

ChatGPT, Gemini, and Claude use preset models. Claude uses Haiku 5.5 with low reasoning effort.
The Actor manages request speed, retries, and deadlines automatically.

#### Filter support by engine

Each engine supports filters differently. The Actor does not present a prompt preference as a guaranteed filter.

| Filter | ChatGPT | Claude | Gemini | Perplexity |
| --- | --- | --- | --- | --- |
| Domain filter | Native | Native | Prompt preference and citation filter | Prompt preference and citation filter |
| Recency | Not supported | Prompt preference | Prompt preference | Prompt preference |
| Location | Native approximate location | Native approximate location | Prompt preference | Prompt preference |
| Search context | Native | Set by Anthropic | Set by Google | Native |

A citation filter removes citations outside your domains. The answer text can still use other sources, and the result includes a warning about this.

### Output example

Each dataset row contains one question and the results from all selected engines. This example is shortened.

```json
{
  "query": "best project management tools for small teams",
  "status": "success",
  "enginesRequested": ["chatgpt", "gemini", "perplexity", "claude"],
  "enginesSucceeded": ["chatgpt", "gemini", "perplexity", "claude"],
  "results": {
    "gemini": {
      "status": "success",
      "aiAnswer": "For small teams, the most popular project management tools are...",
      "citations": [
        {
          "position": 1,
          "url": "https://example.com/project-management-tools",
          "canonicalUrl": "https://example.com/project-management-tools",
          "domain": "example.com",
          "title": "The best project management tools",
          "snippet": "the most popular project management tools"
        }
      ],
      "citationCount": 9,
      "searchPerformed": true,
      "searchQueries": ["best project management software small teams"],
      "tokenUsage": { "input": 411, "output": 383, "total": 1388, "reasoning": 594 },
      "latencyMs": 6120
    },
    "chatgpt": { "status": "success", "aiAnswer": "...", "citationCount": 6 },
    "perplexity": { "status": "success", "aiAnswer": "...", "citationCount": 7 },
    "claude": { "status": "success", "model": "claude-haiku-5-5", "aiAnswer": "...", "citationCount": 5 }
  },
  "crossEngineAnalysis": {
    "overlappingSources": [
      {
        "url": "https://example.com/project-management-tools",
        "domain": "example.com",
        "appearsIn": ["chatgpt", "gemini", "perplexity", "claude"],
        "positions": { "chatgpt": 2, "gemini": 1, "perplexity": 1, "claude": 1 }
      }
    ],
    "totalUniqueSources": 20,
    "totalOverlapping": 1,
    "sourceOverlapRate": 0.05,
    "enginesCompared": ["chatgpt", "gemini", "perplexity", "claude"],
    "engineAgreementSummary": "chatgpt, gemini, perplexity, claude share 5% of unique cited URLs."
  }
}
```

The ChatGPT, Claude, and Perplexity results have the same core fields as the Gemini result.

#### How the source overlap works

The Actor normalizes every cited URL before the comparison.
It removes tracking parameters, fragments, `www.`, and trailing slashes, so the same page counts once.
`sourceOverlapRate` is the number of shared URLs divided by all unique cited URLs.
A high overlap shows that the engines agree on the trusted sources for a question.

### Integrations and API

You can use AI Search Scraper with the [Apify API](https://docs.apify.com/api/v2), the JavaScript and Python API clients, and integrations such as Make, Zapier, n8n, Slack, Google Drive, and webhooks.
The **API** tab on this page shows ready-to-use code for your account.
Use [Apify schedules](https://docs.apify.com/platform/schedules) to track AI answers for your brand every day or week.

### FAQ

#### Do I need an OpenAI, Anthropic, Gemini, or Perplexity API key?

No. The Actor includes the connections to all four engines. You pay only through Apify.

#### Does the Actor use native web search?

Yes. ChatGPT and Claude use native web search. Gemini uses native Google Search grounding.
These are the search features used in their web apps.

Your account personalization, saved memories, past conversations, location, and model settings can affect the answers.
The Actor asks each question independently, without your personal account history.
Answers can also change between runs.

#### Does the Actor confirm that the engine searched the web?

Yes. Each result has `searchPerformed`. If an engine answers without evidence of a web search, the Actor reports the error `SEARCH_NOT_PERFORMED` and does not charge for that result.

#### Can I track my brand in ChatGPT, Claude, Gemini, and Perplexity answers?

Yes. Enter the questions that your customers ask. Then search the `aiAnswer` text for your brand name, and the `citations` for your domain.
Schedule the run to see how your AI search visibility changes over time.

#### Which languages and countries are supported?

You can ask questions in any language that the engines support. Use `userLocation` to request results for a country, region, or city.
ChatGPT and Claude apply the location natively. Gemini and Perplexity receive it as a preference.
See [Filter support by engine](#filter-support-by-engine).

#### What happens if one engine fails?

The other engines still return their answers. The row status is `partial`, and the failed engine shows its error. Failed and skipped engines are free.

#### How many questions can I run?

Up to 500 questions per run. Set `maxQueriesPerRun` to 500, because the default is 50. Use more runs or schedules for larger lists.

#### Is it legal to scrape AI search results?

The Actor sends your questions to AI models and returns their answers and the public web pages they cite. It does not collect data from your accounts.
AI answers can still mention people. Personal data is protected by laws such as the GDPR, so do not collect it without a legitimate reason.
You are responsible for how you use the results. Consult a lawyer if you are not sure about your use case.

### Support and feedback

Report a problem or request a feature in the **Issues** tab. We read every report and use it to improve the Actor.

# Actor input Schema

## `queries` (type: `array`):

One question per line. Each question produces one dataset row.

## `engines` (type: `array`):

Each query runs on the selected engines. The default selects all four: ChatGPT, Gemini, Perplexity, and Claude Haiku.

## `maxQueriesPerRun` (type: `integer`):

Process the first queries up to this limit. Additional queries appear in the summary as truncated.

## `outputFormat` (type: `string`):

The dataset always contains unified JSON rows. CSV also creates a flat file with one row per query and engine.

## `searchContextSize` (type: `string`):

Native for ChatGPT and Perplexity. Gemini and Claude use their own search context.

## `searchRecency` (type: `string`):

Gemini, Perplexity, and Claude use this value as a source preference. ChatGPT does not support this preference.

## `userLocation` (type: `object`):

Default: New York City, United States. Use a two-letter country code, city, region, and timezone. Use {} for no location preference.

## `domainFilter` (type: `object`):

Use allowed or blocked domains. Maximum 20 domains with Perplexity, or 100 without Perplexity. Gemini and Perplexity filter the returned citations.

## `perplexityModel` (type: `string`):

Sonar Pro is the default. ChatGPT, Gemini, and Claude use fixed models. Claude uses Haiku 5.5 with low reasoning effort.

## `includeRelatedQuestions` (type: `boolean`):

Include related questions when an engine returns them.

## `includeAllSources` (type: `boolean`):

Include sources that an engine consulted but did not cite, when the engine reports them.

## `maxCitationsPerEngine` (type: `integer`):

Maximum citations per engine

## `deduplicateSources` (type: `boolean`):

Remove duplicate citations

## Actor input object example

```json
{
  "queries": [
    "best project management tools for small teams"
  ],
  "engines": [
    "chatgpt",
    "gemini",
    "perplexity",
    "claude"
  ],
  "maxQueriesPerRun": 50,
  "outputFormat": "json",
  "searchContextSize": "medium",
  "searchRecency": "any",
  "userLocation": {
    "country": "US",
    "city": "New York",
    "region": "New York",
    "timezone": "America/New_York"
  },
  "domainFilter": {
    "allowed": [],
    "blocked": []
  },
  "perplexityModel": "sonar-pro",
  "includeRelatedQuestions": true,
  "includeAllSources": false,
  "maxCitationsPerEngine": 20,
  "deduplicateSources": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "best project management tools for small teams"
    ],
    "userLocation": {
        "country": "US",
        "city": "New York",
        "region": "New York",
        "timezone": "America/New_York"
    },
    "domainFilter": {
        "allowed": [],
        "blocked": []
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("stefanie-rink/ai-search-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["best project management tools for small teams"],
    "userLocation": {
        "country": "US",
        "city": "New York",
        "region": "New York",
        "timezone": "America/New_York",
    },
    "domainFilter": {
        "allowed": [],
        "blocked": [],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("stefanie-rink/ai-search-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "best project management tools for small teams"
  ],
  "userLocation": {
    "country": "US",
    "city": "New York",
    "region": "New York",
    "timezone": "America/New_York"
  },
  "domainFilter": {
    "allowed": [],
    "blocked": []
  }
}' |
apify call stefanie-rink/ai-search-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,stefanie-rink/ai-search-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/afHYa1aDysPPHQchC/builds/a9iS7bZOa76bjVdyI/openapi.json