# ChatGPT Scraper (`rl1987/chatgpt-scraper`) Actor

Send prompts to chatgpt.com and get the answers back as structured JSON — text, markdown, HTML, citations and sources.

- **URL**: https://apify.com/rl1987/chatgpt-scraper.md
- **Developed by:** [R.L.](https://apify.com/rl1987) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 chatgpt searches

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## ChatGPT Scraper

Ask ChatGPT anything at scale and get structured JSON back — the answer as plain text, markdown
and HTML, plus the sources it cited when it searched the web.

No OpenAI API key, no ChatGPT account, no browser. This Actor talks to chatgpt.com's own
logged-out web interface over plain HTTP, which makes it fast and cheap: **20 prompts answered
in 50 seconds for 0.055 compute units**.

### What you can use it for

- **AI visibility / brand monitoring** — track whether ChatGPT recommends you, and which sources
  it cites when it does.
- **SEO and citation research** — see which domains ChatGPT pulls from for a topic, with titles
  and publication dates.
- **Market and competitor research** — ask the same question from different countries and compare
  the answers.
- **Content and product research** — batch hundreds of prompts and get a clean dataset instead of
  copy-pasting from a chat window.

### Input

| Field | Type | Description |
|---|---|---|
| `prompts` | string\[] | **Required.** One prompt per conversation, up to 4096 characters each. |
| `country` | string | Answer as if browsing from this country. Picked from a dropdown; needs a proxy to have any effect. |
| `additionalPrompt` | string | A follow-up asked in the same conversation. Its answer is the one saved. |
| `requireSources` | boolean | Flag answers that cite no web sources as failed. |
| `maxConcurrency` | integer | Prompts answered at once. Default 5, max 20. |
| `maxRetries` | integer | Retries per prompt, each from a fresh session and exit node. Default 3. |
| `proxyConfiguration` | object | Residential proxy strongly recommended — see [Proxies](#proxies). |

```json
{
  "prompts": [
    "Top hotels in New York",
    "What are the biggest business trends to watch in the next five years?"
  ],
  "country": "us",
  "additionalPrompt": "Which of these are pet-friendly?",
  "maxConcurrency": 5,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["RESIDENTIAL"] }
}
```

Every prompt in a run uses the same `country` and `additionalPrompt`. To compare countries, run
one task per country.

### Output

One dataset row per prompt.

```json
{
  "url": "https://chatgpt.com/?q=Name+exactly+three+rivers+in+Lithuania",
  "prompt": "Name exactly three rivers in Lithuania",
  "answer_text": "Nemunas\nNeris\nVenta",
  "answer_text_markdown": "- Nemunas\n- Neris\n- Venta",
  "answer_html": "<ul>\n<li>Nemunas</li>\n<li>Neris</li>\n<li>Venta</li>\n</ul>\n",
  "model": null,
  "web_search_triggered": false,
  "citations": [],
  "references": [],
  "search_sources": [],
  "links_attached": [],
  "country": null,
  "prompt_sent_at": "2026-09-10T13:57:19.000Z",
  "index": 0,
  "conversation_id": "6aa2b73f-4cec-83ea-a725-21ed1554e4f0"
}
```

| Field | Description |
|---|---|
| `answer_text` / `answer_text_markdown` / `answer_html` | The same answer as plain text, markdown and rendered HTML. |
| `web_search_triggered` | Whether ChatGPT went to the web for this answer. |
| `citations` | Inline sources attached to individual claims: `{url, title, attribution, position, published_at?, is_supporting?}`. |
| `references` | The answer-level source list behind ChatGPT's "Sources" button, same shape. A URL can appear in both — ChatGPT shows them as separate things. |
| `search_sources` | Every source touched, deduplicated: `{url, title, favicon}`. |
| `links_attached` | Links written into the answer body: `{url, text, position}`. |
| `index` | Position of the prompt in your input. |
| `conversation_id` | ChatGPT's id for the conversation the answer came from. |
| `model` | Always `null` — the logged-out interface never names the model it used. |

Prompts that fail after every retry are still saved, carrying `error` and `error_code` instead
of an answer, so a run never silently drops one.

Answers are pushed the moment they land, so with concurrency on, rows arrive out of order. Sort
by `index` to get your input order back.

### Pricing

**$2 per 1000 searches** ($0.002 each), charged per answer actually delivered. Prompts that fail
cost nothing.

### Proxies

ChatGPT rate limits datacenter IPs hard: they start timing out and then get blocked by Cloudflare
after a few dozen requests. Run this on **residential** proxies, which is the default
configuration. `country` works by choosing the proxy's exit node, so it does nothing without one.

### Speed and reliability

Measured on 20 mixed prompts through Apify residential proxies:

| Setting | Wall clock | Compute units |
|---|---|---|
| `maxConcurrency: 1` | 278s | 0.308 |
| `maxConcurrency: 5` | 50s | 0.055 |

Individual prompts take roughly 8-25 seconds depending on answer length and whether ChatGPT
searches the web.

Transient failures happen — a dead proxy exit, an HTTP 503 from ChatGPT, or ChatGPT deciding a
particular session must sign in. Each is retried from a fresh session and a different exit node
with growing backoff, and in testing every one recovered: 60 out of 60 prompts answered across
runs at both concurrency settings.

### How it works

ChatGPT's logged-out interface is a server-rendered web app, so a whole turn can run over plain
HTTP requests: the Actor completes the site's anti-bot handshake (including its proof-of-work
challenge), submits the prompt, and reads the streamed answer, which arrives with markdown, HTML
and source metadata already attached. No browser is started, which is why runs are quick and
cheap compared with browser-driven scrapers.

### Limitations

- One country per run.
- Answers come from the logged-out ChatGPT, so there is no account history, no file uploads, no
  image generation, and no choice of model.
- ChatGPT is non-deterministic: the same prompt can return different answers and cite different
  sources on different runs.
- Answers can be wrong or out of date. Treat the cited sources as the ground truth, not the prose.

# Actor input Schema

## `prompts` (type: `array`):

Prompts to send to ChatGPT, one conversation each, max 4096 characters. Every prompt uses the country and follow-up set below.

## `country` (type: `string`):

Answer as if browsing from this country — it routes the proxy exit node, which is what ChatGPT actually sees. Needs a proxy to have any effect.

## `additionalPrompt` (type: `string`):

Optional follow-up sent in the same conversation after each prompt. The follow-up's answer is what gets saved.

## `requireSources` (type: `boolean`):

Mark a result as failed when the answer cites no web sources.

## `maxRetries` (type: `integer`):

How many times to retry a prompt (with a fresh session) before giving up.

## `maxConcurrency` (type: `integer`):

How many prompts to answer at once. Each one gets its own session and proxy exit node, so raising this shortens a run roughly proportionally. Lower it to 1 if a provider starts rate limiting you.

## `proxyConfiguration` (type: `object`):

Strongly recommended. Datacenter IPs get throttled by Cloudflare after a few dozen requests; residential exits do not.

## Actor input object example

```json
{
  "prompts": [
    "Top hotels in New York"
  ],
  "country": "",
  "requireSources": false,
  "maxRetries": 3,
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `answers` (type: `string`):

Every prompt's answer, with its citations and sources.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "prompts": [
        "Top hotels in New York"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("rl1987/chatgpt-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "prompts": ["Top hotels in New York"] }

# Run the Actor and wait for it to finish
run = client.actor("rl1987/chatgpt-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "prompts": [
    "Top hotels in New York"
  ]
}' |
apify call rl1987/chatgpt-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rl1987/chatgpt-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/hgb7kszETY52fsSYp/builds/uicb9QFVsQYN0WTMv/openapi.json