Gemini Scraper – AI Answers, Sources & Citations

Gemini Scraper returns one structured dataset record per processed prompt, with the answer in Markdown and plain text, ordered web sources, language, exit-IP country, optional brand detection, a UTC timestamp, and an explicit status. It collects answers from the anonymous Gemini web interface without requiring a Google login or Gemini API key.

What is this Gemini Scraper for?

Use it to build prompt-and-answer datasets for generative engine optimization (GEO), SEO research, brand monitoring, competitive analysis, and content research. Instead of copying answers and citations manually, you submit a list of prompts and receive consistent records that can be exported, compared, or processed by another application.

This Actor is intended for:

• GEO and SEO teams comparing the answers and cited domains returned for recurring search-style prompts.
• Brand and marketing teams checking whether a named brand appears in each answer and inspecting the sources returned alongside it.
• Content researchers collecting Gemini's answers in both Markdown and plain text for later analysis.
• Developers and analysts integrating prompt results into Python workflows, scheduled Apify runs, or downstream datasets.

Each prompt is handled as a separate, single-turn request. The Actor does not maintain a conversation, choose a Gemini model, upload files, or generate media. It records what Gemini returned at scrape time; it does not calculate search rankings, visibility scores, or sentiment.

Abridged output example

The values below illustrate the dataset structure; answer_markdown and answer_text are shortened with an ellipsis. Separately, the five-prompt Apify control run wdHOhXNVGmr3DJxtj, recorded in BATTERIE.json on November 9, 2026, finished successfully with 5/5 complete answers and 0 blocked prompts in 39.905 seconds. This is a measured five-prompt run, not a guarantee of latency or future availability.

AI-generated output is not verified by this Actor; check important claims independently before relying on it.

{
  "prompt": "best crm for startups",
  "answer_markdown": "Choosing the best CRM for a startup depends heavily on your team's size, budget, and primary motion (e.g., high-volume inbound, outbound sales, or relationship-driven B2B SaaS). \n\nThe top-rated CRM options for startups are categorized below by what they do best:\n\n…",
  "answer_text": "Choosing the best CRM for a startup depends heavily on your team's size, budget, and primary motion (e.g., high-volume inbound, outbound sales, or relationship-driven B2B SaaS). \n\nThe top-rated CRM options for startups are categorized below by what they do best:\n\n…",
  "sources": [
    {
      "url": "https://attio.com/f/best-crm-for-startups",
      "title": "Best CRMs for startups in 2026 - Attio",
      "domain": "attio.com",
      "position": 1
    },
    {
      "url": "https://blog.salesflare.com/best-crm-for-startups",
      "title": "9 Best CRMs for Startups in 2026 - Salesflare Blog",
      "domain": "blog.salesflare.com",
      "position": 2
    },
    {
      "url": "https://www.techradar.com/best/the-best-crm-for-startups",
      "title": "Best CRM for startups of 2026 - TechRadar",
      "domain": "techradar.com",
      "position": 3
    },
    {
      "url": "https://smashsend.com/blog/best-crm-for-saas-startups",
      "title": "The 12 Best CRM for SaaS Startups in 2026: An In-Depth Guide",
      "domain": "smashsend.com",
      "position": 4
    },
    {
      "url": "https://fluidcrm.io/blog/best-crms-for-startups/",
      "title": "9 Best CRMs for Startups In 2026 (Priced and Ranked) - Fluid CRM",
      "domain": "fluidcrm.io",
      "position": 5
    },
    {
      "url": "https://www.expertmarket.com/crm/compare-pricing",
      "title": "Compare CRM Pricing and Costs in 2026 - Expert Market",
      "domain": "expertmarket.com",
      "position": 6
    },
    {
      "url": "https://www.business.com/categories/crm-software/",
      "title": "The Best CRM Software of 2026 - Business.com",
      "domain": "business.com",
      "position": 7
    },
    {
      "url": "https://monday.com/blog/crm-and-sales/crm-for-start-ups/",
      "title": "13 best CRM platforms for startups in 2026 - Monday.com",
      "domain": "monday.com",
      "position": 8
    },
    {
      "url": "https://www.pcmag.com/picks/the-best-crm-software",
      "title": "The Best CRM Software We've Tested for 2026 | PCMag",
      "domain": "pcmag.com",
      "position": 9
    }
  ],
  "sources_count": 9,
  "target_brand_mentioned": true,
  "country": "US",
  "language": "en",
  "status": "ok",
  "scraped_at": "2026-11-09T05:51:13.367946Z"
}

Why choose this Actor?

Structured answers and citations

You receive both answer_markdown and answer_text, so you can preserve headings and lists or work with a plain-text rendition. Sources remain structured objects with url, title, domain, and position, rather than being flattened into a string. Highlight fragments are removed, duplicate www variants are consolidated, and positions are assigned after deduplication.

A complete answer does not necessarily include citations. If Gemini returns no usable web sources, sources is an empty array and sources_count is 0.

Brand checks without another processing step

Set targetBrand to add a case-insensitive whole-brand match in target_brand_mentioned. Leave it empty to return null. This is a literal brand-name check in the answer text, not sentiment analysis, entity disambiguation, or a multi-brand comparison; you can run separate inputs or analyze the exported text for additional names.

Transparent outcomes and measured tests

Every processed prompt has an ok, empty, blocked, or error status. Unsuccessful rows remain visible in the dataset instead of silently disappearing. The RUN_SUMMARY record adds status counts, errors by type, attempts, transfer volume, elapsed time, and budget state.

A direct-IP prototype returned 30 complete answers out of 30 prompts, with a median processing time of 4.97 seconds per prompt, and all 30 contained structured sources. Separately, the Apify control run wdHOhXNVGmr3DJxtj delivered 5/5 complete answers with no recorded problems. These are small measured samples: they do not establish high-volume rate limits, validate every proxy configuration, or guarantee that Gemini's internal endpoint will remain available.

No account or browser setup

The Actor uses sequential HTTP requests with a Chrome TLS fingerprint. It does not launch a browser and does not ask for your Google credentials, account cookies, a Gemini API key, or a paid Gemini API connection. Anonymous answers can nevertheless differ from those shown to a signed-in user.

Pay for complete answers

The primary result event costs $0.005 per complete answer, exactly $5.00 per 1,000 results. Only rows with status: "ok" trigger this event; empty, blocked, and error rows do not. Dataset delivery and charging are performed together through the Apify SDK, and the Actor checks the remaining run budget before each prompt.

Standard Apify platform usage and optional proxy costs are separate. Apify's standard synthetic Actor-start event may also appear in billing. The Actor stops cleanly when the configured maximum run cost cannot cover another result.

Frequently asked questions

Do I need a Google account or Gemini API key?

No. The current implementation uses anonymous Gemini web requests and does not read your Google account cookies. No Gemini API key is required. Because the web endpoint is internal and undocumented, Google can change it or require authentication without notice.

Is this the official Gemini API?

No. This Actor collects answers from the Gemini web interface; it is not a wrapper around the official Gemini API. Consequently, it has no model selector, official API response schema, multi-turn history, file upload, or media-generation controls.

What exactly am I charged for?

One complete, non-empty answer with status: "ok" costs $0.005, equivalent to $5.00 per 1,000 results. Empty answers, blocked prompts, transport or parsing errors, and responses matching the implemented short-safety-refusal patterns do not trigger the result charge. Platform usage, the possible standard Actor-start event, and any proxy usage are separate from that result price.

Can I track whether Gemini mentions my brand?

Yes. Set targetBrand to the brand or product name you want to find. The Actor performs a case-insensitive match with word boundaries and returns true or false; it returns null when no target is supplied. It does not infer misspellings, aliases, sentiment, or competitor relationships.

Are sources and citations always included?

No. The Actor preserves and deduplicates the structured sources that Gemini returns, but it does not invent citations or force web grounding. A complete answer can have an empty sources array. Source titles, URLs, and domains are output data and should be checked before being used as factual evidence.

Can I select a country or language?

There is no dedicated country or language selector. The prompt language and exit IP can influence the answer. language is the language code reported by Gemini, while country is the ISO 3166-1 alpha-2 country detected for the exit IP; it is not proof of the answer's geographic targeting and can be null if detection fails.

Does proxy support provide reliable geo-targeting?

You can supply an Apify proxy configuration or custom HTTP proxy URLs, and the same proxy session is used for Gemini and country detection. However, proxy behavior was not validated by the direct-IP battery, and the tested cloud run used a datacenter IP. Treat localization and proxy performance as things to test for your own prompts. Proxy usage may incur additional Apify charges.

What happens when Gemini blocks or rate-limits a request?

The Actor retries network failures, empty responses, blocking pages, and responses matching the implemented short-safety-refusal patterns up to three attempts in total. HTTP 429 uses longer waits and honors a numeric Retry-After value up to the implemented cap; HTTP 403, 408, and 5xx use increasing waits, while 404 is not retried.

A run with no complete answer finishes as failed. A run where more than half of processed prompts are blocked also finishes as failed, even if some answers were delivered. Any delivered ok rows remain subject to the stated result price.

Will the answers match my logged-in Gemini session?

Not necessarily. Anonymous answers can differ by session, location, time, prompt wording, and changes on Google's side. The scraped_at timestamp lets you compare runs, but it does not make different runs identical or reproducible. AI-generated answers can also be inaccurate, so verify important claims independently.

Can I schedule recurring prompt monitoring?

You can run the same input repeatedly with Apify scheduling and compare the timestamped dataset rows. The Actor provides collection and brand matching; historical dashboards, alerts, and visibility scoring must be built in your downstream workflow.

Is this Actor affiliated with Google, and is automated access allowed?

No affiliation with Google is claimed, and there are no affiliate links in this document. Automated access may conflict with Google's terms. The Actor does not bypass CAPTCHAs or anti-bot mechanisms. You are responsible for reviewing the applicable terms and law for your use case.

Settings, output fields, and API

Input settings

• prompts (required): an array of prompt strings, one potential result row per prompt. The schema accepts up to 10,000 strings. Empty prompts and exact duplicates after trimming are removed before processing.
• maxItems (optional): caps the deduplicated prompt list. It accepts integers from 1 to 10,000 and defaults to 1,000; the prefilled example uses 5. A budget stop can result in fewer processed prompts.
• targetBrand (optional): the literal brand or product name used for target_brand_mentioned. The default is empty, which produces null; the prefill uses HubSpot.
• proxyConfiguration (optional): disabled by default and in the prefill. Enable Apify proxy or provide custom proxy URLs through the standard Apify proxy field. Separate proxy charges can apply.

Example input:

{
  "prompts": [
    "best crm for startups",
    "best project management tool for agencies",
    "best running shoes for flat feet",
    "top vpn for streaming",
    "cheapest web hosting for wordpress"
  ],
  "maxItems": 5,
  "targetBrand": "HubSpot",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Output fields

Every processed row contains exactly these ten fields:

• prompt: the trimmed prompt sent to Gemini.
• answer_markdown: the cleaned complete answer in Markdown, or an empty string for non-ok rows.
• answer_text: a plain-text rendition of the Markdown answer, or an empty string for non-ok rows.
• sources: ordered objects containing url, title, domain, and one-based position; empty for non-ok rows and for answers without sources.
• sources_count: the number of entries in sources.
• target_brand_mentioned: true, false, or null as described above.
• country: the exit-IP country code or null.
• language: Gemini's reported language code, or an empty string if none is reported or processing fails.
• status: ok, empty, blocked, or error; only ok is charged as a result.
• scraped_at: the row's UTC ISO 8601 timestamp.

The default key-value store also contains RUN_SUMMARY with prompts, processed, billedResults, statuses, errorsByType, attempts, transferMB, country, elapsedSeconds, stoppedByBudget, and aborted.

Python API

Install the official Apify API client with pip install apify-client, set your token in the APIFY_TOKEN environment variable, and pass the same list of prompts used in the JSON input:

import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

prompts = [
    "best crm for startups",
    "best project management tool for agencies",
    "best running shoes for flat feet",
]

run = client.actor("kazkn/gemini-ai-answers-scraper").call(
    run_input={
        "prompts": prompts,
        "maxItems": len(prompts),
        "targetBrand": "HubSpot",
        "proxyConfiguration": {"useApifyProxy": False},
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["prompt"], item["status"], item["sources_count"])

Keep your API token in the environment or a secret manager; do not commit it to source code. The returned run object contains the dataset and key-value store IDs needed to retrieve results and RUN_SUMMARY.

API URLs

Use an Authorization: Bearer YOUR_APIFY_TOKEN header for authenticated requests. The relevant endpoints are:

• Start a run: POST https://api.apify.com/v2/acts/kazkn~gemini-ai-answers-scraper/runs
• Get run details: GET https://api.apify.com/v2/actor-runs/{runId}
• Read dataset items: GET https://api.apify.com/v2/datasets/{datasetId}/items
• Read the run summary: GET https://api.apify.com/v2/key-value-stores/{storeId}/records/RUN_SUMMARY

For dataset items, add ?format=json or another Apify-supported export format. Replace {runId}, {datasetId}, and {storeId} with the values returned for your run rather than using IDs from the example above.