ChatGPT, Claude, Gemini & Perplexity AI Search Scraper

A paid Apify plan is required. Free-plan runs stop before AI requests. Upgrade your Apify plan to use this Actor.

AI Search Scraper sends your questions to ChatGPT Search, Claude Haiku, Google Gemini, and Perplexity Sonar Pro in one run. ChatGPT and Claude use native web search. Gemini uses native Google Search grounding. Claude uses Haiku 5.5 with low reasoning effort. It returns each engine's AI answer, cited sources and URLs, search queries, and the source overlap between engines in a single dataset. Use it for GEO (generative engine optimization), AEO (answer engine optimization), brand visibility tracking in AI search, and citation research.

• ✅ No API keys needed. The Actor includes the connections to all four engines.
• ✅ Questions in any language, with optional location, recency, and domain preferences.
• ✅ Export to JSON, CSV, Excel, or HTML, or connect the results to your tools through the Apify API.

What does AI Search Scraper do?

AI Search Scraper combines a ChatGPT scraper, Claude scraper, Gemini scraper, and Perplexity scraper. For each question, it:

• 🔍 Asks ChatGPT, Claude, Gemini, and Perplexity in parallel, with web search turned on.
• 📝 Extracts the full AI-generated answer text.
• 🔗 Collects the citations with URL, domain, title, and snippet.
• 🧭 Records the search queries that the engine sent to the web, when the engine reports them.
• ❓ Keeps related questions when the engine suggests them.
• 📊 Compares the engines and shows the sources they all cite and the sources that only one engine cites.
• 🌐 Converts Google grounding redirect links into the actual source URLs.

Why scrape AI search engines?

More people now get answers from ChatGPT, Claude, Gemini, and Perplexity instead of a list of blue links. If an AI answer does not mention your brand or cite your site, those people may never see you. AI Search Scraper shows you what each AI search engine says and which websites it trusts.

Use cases

• Brand visibility in AI search: See whether ChatGPT, Claude, Gemini, and Perplexity mention your brand, product, or competitors.
• GEO and AEO monitoring: Track which pages AI engines cite, and measure your progress after you change content.
• Competitor research: Find the domains and pages that AI answers cite for your market.
• Content and SEO strategy: Read the search queries that AI engines generate, and find content gaps.
• Digital PR and link building: Find the publications that AI engines cite most, and target them.
• AI answer comparison and research: Compare how four AI search engines answer the same question, in any language and location.
• Scheduled monitoring: Run the same questions every day or week with Apify schedules, and track changes over time.

What data can you extract from ChatGPT, Claude, Gemini, and Perplexity?

Data	Description
AI answer	The full answer text from each engine.
Citations	URL, canonical URL, domain, title, snippet, and position of each cited source.
Search queries	The web search queries that the engine used, when the engine reports them.
Related questions	Follow-up questions, when the engine returns them.
All consulted sources	Optional. Sources that the engine read but did not cite.
Source overlap	Sources that two or more engines cite, unique sources per engine, and the overlap rate.
Model and usage	Model, token usage, latency, and attempts.
Status and warnings	Success, error, timeout, or skipped status for each engine, with a clear reason.
Gemini extras	Grounding supports that link answer text to sources, and Google Search suggestions.

How to scrape ChatGPT, Claude, Gemini, and Perplexity answers

1. Open the Actor in Apify Console with a paid Apify plan.
2. Enter one or more questions in Search queries. For example: best CRM for small businesses.
3. Select ChatGPT, Claude, Gemini, Perplexity, or any combination. The default selects all four.
4. Optional: change the default New York City location, source recency, or allowed or blocked domains.
5. Click Start.
6. Download the results as JSON, CSV, Excel, or HTML, or read them through the API.

Each question produces one dataset row that contains the results from all selected engines.

How much does it cost to scrape AI search answers?

AI Search Scraper uses pay-per-event pricing. You pay for each successful engine answer. You do not need your own OpenAI, Anthropic, Google, or Perplexity account.

What you run	Price
One question on one engine	$0.05
One question on all four engines	about $0.13
100 questions on all four engines	about $13.33
Failed or skipped engine results	Free

With several engines, each successful engine result costs $0.0333. If only two of four engines succeed, you pay for two.

Input example

{
  "queries": [
    "best project management tools for small teams",
    "is Notion or Asana better for startups"
  ],
  "engines": ["chatgpt", "gemini", "perplexity", "claude"],
  "perplexityModel": "sonar-pro",
  "userLocation": { "country": "US" },
  "includeAllSources": true
}

Input options

Option	Default	Description
queries	—	The questions to ask. maxQueriesPerRun sets how many run.
engines	All four	chatgpt, gemini, perplexity, and claude. Select one engine or any combination.
perplexityModel	Sonar Pro	sonar, sonar-pro, or sonar-reasoning-pro.
userLocation	New York City, US	Two-letter country code, city, region, and IANA timezone. Use {} for no location preference.
searchRecency	any	hour, day, week, month, or any.
domainFilter	None	allowed or blocked domains, not both. Up to 20 domains with Perplexity, or 100 without it.
searchContextSize	medium	low, medium, or high search depth for ChatGPT and Perplexity.
includeAllSources	false	Include sources that the engine read but did not cite.
includeRelatedQuestions	true	Keep the related questions.
maxCitationsPerEngine	20	Maximum citations per engine.
maxQueriesPerRun	50	Maximum questions per run. Limit: 500.
outputFormat	json	csv also saves RESULTS_CSV in the key-value store, with one row per question and engine.

ChatGPT, Gemini, and Claude use preset models. Claude uses Haiku 5.5 with low reasoning effort. The Actor manages request speed, retries, and deadlines automatically.

Filter support by engine

Each engine supports filters differently. The Actor does not present a prompt preference as a guaranteed filter.

Filter	ChatGPT	Claude	Gemini	Perplexity
Domain filter	Native	Native	Prompt preference and citation filter	Prompt preference and citation filter
Recency	Not supported	Prompt preference	Prompt preference	Prompt preference
Location	Native approximate location	Native approximate location	Prompt preference	Prompt preference
Search context	Native	Set by Anthropic	Set by Google	Native

A citation filter removes citations outside your domains. The answer text can still use other sources, and the result includes a warning about this.

Output example

Each dataset row contains one question and the results from all selected engines. This example is shortened.

{
  "query": "best project management tools for small teams",
  "status": "success",
  "enginesRequested": ["chatgpt", "gemini", "perplexity", "claude"],
  "enginesSucceeded": ["chatgpt", "gemini", "perplexity", "claude"],
  "results": {
    "gemini": {
      "status": "success",
      "aiAnswer": "For small teams, the most popular project management tools are...",
      "citations": [
        {
          "position": 1,
          "url": "https://example.com/project-management-tools",
          "canonicalUrl": "https://example.com/project-management-tools",
          "domain": "example.com",
          "title": "The best project management tools",
          "snippet": "the most popular project management tools"
        }
      ],
      "citationCount": 9,
      "searchPerformed": true,
      "searchQueries": ["best project management software small teams"],
      "tokenUsage": { "input": 411, "output": 383, "total": 1388, "reasoning": 594 },
      "latencyMs": 6120
    },
    "chatgpt": { "status": "success", "aiAnswer": "...", "citationCount": 6 },
    "perplexity": { "status": "success", "aiAnswer": "...", "citationCount": 7 },
    "claude": { "status": "success", "model": "claude-haiku-5-5", "aiAnswer": "...", "citationCount": 5 }
  },
  "crossEngineAnalysis": {
    "overlappingSources": [
      {
        "url": "https://example.com/project-management-tools",
        "domain": "example.com",
        "appearsIn": ["chatgpt", "gemini", "perplexity", "claude"],
        "positions": { "chatgpt": 2, "gemini": 1, "perplexity": 1, "claude": 1 }
      }
    ],
    "totalUniqueSources": 20,
    "totalOverlapping": 1,
    "sourceOverlapRate": 0.05,
    "enginesCompared": ["chatgpt", "gemini", "perplexity", "claude"],
    "engineAgreementSummary": "chatgpt, gemini, perplexity, claude share 5% of unique cited URLs."
  }
}

The ChatGPT, Claude, and Perplexity results have the same core fields as the Gemini result.

How the source overlap works

The Actor normalizes every cited URL before the comparison. It removes tracking parameters, fragments, www., and trailing slashes, so the same page counts once. sourceOverlapRate is the number of shared URLs divided by all unique cited URLs. A high overlap shows that the engines agree on the trusted sources for a question.

Integrations and API

You can use AI Search Scraper with the Apify API, the JavaScript and Python API clients, and integrations such as Make, Zapier, n8n, Slack, Google Drive, and webhooks. The API tab on this page shows ready-to-use code for your account. Use Apify schedules to track AI answers for your brand every day or week.

FAQ

Do I need an OpenAI, Anthropic, Gemini, or Perplexity API key?

No. The Actor includes the connections to all four engines. You pay only through Apify.

Does the Actor use native web search?

Yes. ChatGPT and Claude use native web search. Gemini uses native Google Search grounding. These are the search features used in their web apps.

Your account personalization, saved memories, past conversations, location, and model settings can affect the answers. The Actor asks each question independently, without your personal account history. Answers can also change between runs.

Does the Actor confirm that the engine searched the web?

Yes. Each result has searchPerformed. If an engine answers without evidence of a web search, the Actor reports the error SEARCH_NOT_PERFORMED and does not charge for that result.

Can I track my brand in ChatGPT, Claude, Gemini, and Perplexity answers?

Yes. Enter the questions that your customers ask. Then search the aiAnswer text for your brand name, and the citations for your domain. Schedule the run to see how your AI search visibility changes over time.

Which languages and countries are supported?

You can ask questions in any language that the engines support. Use userLocation to request results for a country, region, or city. ChatGPT and Claude apply the location natively. Gemini and Perplexity receive it as a preference. See Filter support by engine.

What happens if one engine fails?

The other engines still return their answers. The row status is partial, and the failed engine shows its error. Failed and skipped engines are free.

How many questions can I run?

Up to 500 questions per run. Set maxQueriesPerRun to 500, because the default is 50. Use more runs or schedules for larger lists.

Is it legal to scrape AI search results?

The Actor sends your questions to AI models and returns their answers and the public web pages they cite. It does not collect data from your accounts. AI answers can still mention people. Personal data is protected by laws such as the GDPR, so do not collect it without a legitimate reason. You are responsible for how you use the results. Consult a lawyer if you are not sure about your use case.

Support and feedback

Report a problem or request a feature in the Issues tab. We read every report and use it to improve the Actor.