Google Search Results Scraper

Google Search Results Scraper — Organic, Ads and AI Overview Data

Google Search Results Scraper runs real Google searches and returns organic results, paid ads, shopping products, People Also Ask questions, related searches, and an optional AI Overview as structured JSON — no HTML parsing required. Feed it plain keywords or full Google Search URLs, layer on site:, intitle:, filetype:, and date-range filters, and get one dataset row per page per query, ready to load into a spreadsheet, database, or LLM pipeline. Start a run from the Actor page below.

What is Google Search Results Scraper?

Google Search Results Scraper is an Apify Actor that fetches Google Search Engine Results Pages (SERPs) over plain HTTP and parses them into typed JSON — organic listings, ads, shopping results, People Also Ask, related searches, and (with your own SerpApi key) AI Overview text. No Google account or login is required; the request path uses only Apify's own proxy network, with no sign-in step anywhere in the code. It's built for SEO and SERP-monitoring teams, marketers doing keyword and content research, and developers or AI engineers who need machine-readable Google search data for a pipeline, dashboard, or agent.

What Google search data is publicly available to scrape?

Everything a logged-out visitor sees on a Google Search results page — organic listings, ads, shopping results, and the collapsed People Also Ask questions — is publicly accessible without a Google account. Only a handful of things sit behind a further gate.

Data category	Publicly available (no login)	Gated behind
Organic results (title, URL, description, sitelinks)	Yes	—
Paid ads / shopping products	Yes, when Google's ad decisioning serves them	Ad volume varies by proxy IP, session, and country — not a login wall
People Also Ask questions	Yes	Expanded answer text — loaded via a click-triggered request, out of reach for a plain HTTP fetch
Related searches & big sitelinks	Yes, when Google renders the block	Absent on many queries — Google's own choice, not access control
AI Overview text	No, by default	Requires your own paid SerpApi key — this Actor holds no AI Overview credentials of its own
Personalized ranking, search history, Discover feed	No	Requires a signed-in Google account

Google Search Results Scraper only returns publicly visible data — what any visitor sees. Nothing behind a Google login wall.

What data can I extract with Google Search Results Scraper?

Every dataset row is one scraped SERP page, carrying the page's search context plus five distinct result arrays — organic, paid, shopping, People Also Ask, and related searches — under one JSON object.

🔎 Query & page identity fields

Field	Description
searchQueryTerm	The final query string Google received, including any site:, intitle:, filetype:, etc. operators the input added
searchQuery.term	Same query string, nested inside the searchQuery object
searchQuery.url	The Google Search URL actually requested
searchQuery.device	DESKTOP or MOBILE, set from the mobileResults input
searchQuery.page	The page number this row represents (1, 2, 3…)
searchQuery.type	Always SEARCH
searchQuery.domain	Always google.com
searchQuery.countryCode	Two-letter country code used for the search, from countryCode
searchQuery.languageCode	Interface language used, from languageCode
searchQuery.locationUule	The UULE location code, if locationUule was set
searchQuery.resultsPerPage	Actual number of organic results collected on this page
url	Same Google Search URL as searchQuery.url
hasNextPage	true if Google exposed a next-page link and maxPagesPerQuery hasn't been reached yet
serpProviderCode	Internal SERP-provider source code ("O")

📊 Result content fields

Field	Description
resultsTotal	Google's own "About X results" count for the query
organicResults[]	Organic listings — title, url, displayedUrl, description, emphasizedKeywords, siteLinks[], productInfo, type, position, and icon (Base64, only when includeIcons is on)
paidResults[]	Google Ads results, same field shape as organicResults[]; only populated when Google serves ads
paidProducts[]	Shopping-panel products — title, displayedUrl, prices[]
suggestedResults[]	relatedQueries re-shaped as result objects — title, url, type, position
relatedQueries[]	Google's real "related searches" chips — title, url
peopleAlsoAsk[]	Real PAA questions — question, title (duplicate of question), answer (often null — see limitations below), url, date (always null)
aiOverview	{ "text": "...", "references": [{ "title", "url" }] } when a valid SerpApi key is supplied and aiMode isn't off; otherwise null

🗂️ Metadata, add-on echoes and snapshots

Field	Description
customData.perplexitySearch	Echoes the perplexitySearch input back unchanged — enablePerplexity, searchRecency, returnImages, returnRelatedQuestions
customData.chatGptSearch	Echoes the chatGptSearch input back unchanged — enableChatGpt
customData.maximumLeadsEnrichmentRecords	Echoes the maximumLeadsEnrichmentRecords input back unchanged
customData.leadsEnrichmentDepartments	Echoes the leadsEnrichmentDepartments input back unchanged
htmlSnapshotUrl	Direct, openable link to the saved HTML in the run's key-value store, when saveHtmlToKeyValueStore is on
html	Full page HTML, added as an extra field only when saveHtml is enabled

🤖 Add-on: Need additional Google data?

Pair this Actor with Google Scholar Scraper for academic search results and citation counts, Google Maps Reviews Scraper for location reviews, or Google Play Scraper for app-store listings — all published by the same Scraper Engine account. Need a lighter, quick-setup SERP scraper instead? See Google Search Results (SERP) Scraper.

How does Google Search Results Scraper differ from Google's Custom Search JSON API?

Google's own Custom Search JSON API returns organic web results from a Programmable Search Engine you configure in advance; Google Search Results Scraper runs the same kind of search a browser would and returns organic results, ads, shopping products, People Also Ask, related searches, and optional AI Overview text, with no search engine to configure first.

Feature	Custom Search JSON API	Google Search Results Scraper
Availability	Closed to new customers — existing users must migrate off it by January 1, 2027	Available to any Apify user today
Setup	Requires a pre-configured Programmable Search Engine plus an API key	Provide search terms — no engine or key to configure
Free quota	100 queries/day free	No free quota — billed per dataset row via Apify's pay-per-event pricing
Results per query	Hard cap: max 10 results per request, and start + num can't exceed 100	No schema-enforced cap on maxPagesPerQuery — pagination stops when Google's own next-page link disappears
Result types	Organic web results only	Organic, paid ads, shopping products, People Also Ask, related searches, sitelinks, optional AI Overview
Pricing beyond free tier	$5 per 1,000 queries, up to 10,000/day	Charged per row_result event — see the Actor's Pricing tab

Use the Custom Search JSON API if you already have a Programmable Search Engine configured and only need organic results within its 100-result ceiling. Use Google Search Results Scraper when you need ads, shopping, People Also Ask, or AI Overview data, more than 100 results per query, or don't want to configure a search engine at all.

How to use Google Search Results Scraper

No coding is required to run this Actor — it's driven entirely from its input form on the Apify platform.

1. Open the Actor's page on the Apify Store and click Try for free (an Apify account is all you need — no Google account).
2. Paste your search terms into queries — one plain keyword phrase or full google.com/search URL per line. This is the only required input.
3. Optionally raise maxPagesPerQuery to collect more than roughly 10 results per query, pick a countryCode / languageCode, or add site:, intitle:, filetype:, or date-range filters from the Advanced search filters section.
4. Click Start and watch rows land in the dataset in real time as each page is scraped.
5. Export results as JSON, CSV, Excel, or the other formats Apify's dataset UI supports, or pull them programmatically via the Apify API.

How to scale to bulk search-result extraction

queries accepts one search term or Google Search URL per line in a single textarea — there's no separate array field, but no line cap either, so hundreds of keywords can run in one call. Each line becomes its own set of dataset rows (one per page, driven by maxPagesPerQuery), and every query in the list is processed in the same run, one after another, without needing to start a new Actor call per keyword.

What can you do with Google search result data?

• 📈 An SEO analyst tracking rankings uses organicResults[].position and organicResults[].url for their target domains to monitor movement across weekly runs.
• 💰 A PPC / ads analyst uses paidResults[] and paidProducts[].prices to benchmark competitor ad copy and shopping prices for a keyword set.
• ✍️ A content strategist mines peopleAlsoAsk[].question and relatedQueries[].title to build topic clusters and FAQ briefs straight from what people actually search.
• 🤖 A GEO/AEO practitioner compares aiOverview.text and aiOverview.references against organicResults to see which sources Google's AI Overview is citing for a target query.
• 🧠 An AI engineer feeds organicResults[].title and organicResults[].description into a RAG pipeline or agent tool as live, search-grounded context instead of relying on a model's stale training data.

How does Google Search Results Scraper handle rate limits and blocking?

Every request goes through Apify Proxy. By default the Actor uses the GOOGLE_SERP proxy group; if a fetch is redirected to Google's /sorry/ block page, comes back with a CAPTCHA form, or shows the "unusual traffic from your computer network" banner, it escalates to a RESIDENTIAL proxy, retries up to 3 times, and then sticks with residential proxies for the rest of the run. Each individual HTTP request is retried up to 3 times on a non-200 response or a network error. When focusOnPaidAds is on and a page shows no ads or shopping products, that same page is re-fetched up to 3 times on the chance Google simply didn't serve ads that attempt.

⚠️ There is no CAPTCHA solver — blocking is handled entirely through proxy escalation and retries, not by defeating a challenge. If a query still fails after all retries, no dataset row is pushed for it (and nothing is charged for it): only pages that parse successfully are pushed and billed under the row_result event.

⬇️ Input

queries is the only required field. Everything else is optional and has a working default.

Parameter	Required	Type	Description
queries	Yes	string	One search term or full Google Search URL per line. Supports Google operators (site:, OR, quoted phrases). Max 32 words per query.
resultsPerPage	No	integer	Passed to Google as num (default 100, min 1, max 100); Google ignores it on a logged-out fetch and returns ~10 organic results per page regardless. Use maxPagesPerQuery for more.
maxPagesPerQuery	No	integer	Max pages to scrape per query (default 1, min 1, no maximum). Each page ≈ 10 results.
aiMode	No	string (enum)	aiModeOff (default), aiModeWithSearchResults, aiModeOnly. Toggles which Google surface is requested and whether the SerpApi-powered aiOverview add-on is attempted.
serpApiKey	No	string (secret)	Your own paid SerpApi key. Required for aiOverview to return anything other than null when aiMode isn't off.
perplexitySearch	No	object	Default {"enablePerplexity": false, "searchRecency": "", "returnImages": false, "returnRelatedQuestions": false}. Stored and echoed back under customData.perplexitySearch — this Actor holds no Perplexity credentials and does not call the Sonar model itself.
chatGptSearch	No	object	Default {"enableChatGpt": false}. Stored and echoed back under customData.chatGptSearch — this Actor holds no OpenAI credentials and does not call a ChatGPT search model itself.
maximumLeadsEnrichmentRecords	No	integer	Default 0, min 0. Stored and echoed back under customData.maximumLeadsEnrichmentRecords for your own downstream enrichment step; no enrichment is performed here.
leadsEnrichmentDepartments	No	array (enum, multi-select)	Default []. 14 department options (c-suite, sales, marketing, engineering-technical, etc.). Stored and echoed back under customData.leadsEnrichmentDepartments; no filtering is performed here.
focusOnPaidAds	No	boolean	Default false. Re-fetches a page up to 3 times when no ads or shopping products were found, using the same proxy as the rest of the run.
countryCode	No	string (enum)	Default "us". Google Search domain / gl country, from a list of ~240 ISO country codes shown in the input UI.
searchLanguage	No	string (enum)	Default none. Restricts results to a language via Google's lr parameter, from ~35 language options.
languageCode	No	string (enum)	Default "en". Google interface language via the hl parameter, from ~80 language options.
locationUule	No	string	Exact-location Google uule parameter.
forceExactMatch	No	boolean	Default false. Wraps the query in quotes for an exact-phrase search.
site	No	string	Adds site:<value>. Takes precedence over relatedToSite if both are set.
relatedToSite	No	string	Adds related:<value>. Ignored if site is also set.
wordsInTitle	No	array of strings	Default [], max 32 items. Adds intitle:"<word>" per entry.
wordsInText	No	array of strings	Default [], max 32 items. Adds intext:"<word>" per entry.
wordsInUrl	No	array of strings	Default [], max 32 items. Adds inurl:"<word>" per entry.
quickDateRange	No	string	e.g. d10, w1, m3, y1 — passed as tbs=qdr:<value>. Avoid combining with beforeDate/afterDate.
beforeDate	No	string (date)	Absolute (2026-05-03) or relative (8 days, 3 months) date, interpreted in UTC.
afterDate	No	string (date)	Same format as beforeDate.
fileTypes	No	array (enum, multi-select)	Default [], max 10 items. 40+ file-type codes (pdf, csv, docx, mkv, etc.), combined with OR.
mobileResults	No	boolean	Default false. Sends a mobile User-Agent and sets searchQuery.device to MOBILE.
includeUnfilteredResults	No	boolean	Default false. Includes lower-quality results Google normally filters out.
saveHtml	No	boolean	Default false. Adds the raw page HTML to the dataset row under html.
saveHtmlToKeyValueStore	No	boolean	Default true. Saves page HTML to the key-value store and links it via htmlSnapshotUrl.
includeIcons	No	boolean	Default false. Adds Base64 favicon data to result icons when found.
proxyConfiguration	No	object	Default {"useApifyProxy": true, "apifyProxyGroups": ["GOOGLE_SERP"]}. Always uses a proxy; falls back to RESIDENTIAL on a detected block.

Example input

{
  "queries": "best wireless headphones 2026\nsite:reddit.com best noise cancelling headphones",
  "resultsPerPage": 100,
  "maxPagesPerQuery": 3,
  "aiMode": "aiModeOff",
  "serpApiKey": "",
  "perplexitySearch": {
    "enablePerplexity": false,
    "searchRecency": "",
    "returnImages": false,
    "returnRelatedQuestions": false
  },
  "chatGptSearch": {
    "enableChatGpt": false
  },
  "maximumLeadsEnrichmentRecords": 0,
  "leadsEnrichmentDepartments": [],
  "focusOnPaidAds": true,
  "countryCode": "us",
  "searchLanguage": "en",
  "languageCode": "en",
  "locationUule": "",
  "forceExactMatch": false,
  "site": "",
  "relatedToSite": "",
  "wordsInTitle": [],
  "wordsInText": [],
  "wordsInUrl": [],
  "quickDateRange": "",
  "beforeDate": "",
  "afterDate": "",
  "fileTypes": [],
  "mobileResults": false,
  "includeUnfilteredResults": false,
  "saveHtml": false,
  "saveHtmlToKeyValueStore": true,
  "includeIcons": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["GOOGLE_SERP"]
  }
}

⬆️ Output

Each run writes typed, normalized JSON rows to the default dataset — one row per scraped page, per query, with a consistent field set across runs. Export as JSON, CSV, Excel, XML, or RSS directly from the dataset, or pull it through the Apify API. The default dataset view surfaces searchQueryTerm, resultsTotal, relatedQueries, aiOverview, paidResults, paidProducts, organicResults, and peopleAlsoAsk as columns; every field documented above is present on the underlying row regardless of which view is showing.

Example output

{
  "searchQuery": {
    "term": "best wireless headphones 2026",
    "url": "https://www.google.com/search?q=best+wireless+headphones+2026",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "best wireless headphones 2026",
  "url": "https://www.google.com/search?q=best+wireless+headphones+2026",
  "hasNextPage": true,
  "serpProviderCode": "O",
  "resultsTotal": 41200000,
  "relatedQueries": [
    { "title": "best noise cancelling headphones 2026", "url": "https://www.google.com/search?q=best+noise+cancelling+headphones+2026&hl=en&gl=us" }
  ],
  "paidResults": [],
  "paidProducts": [
    { "title": "Sony WH-1000XM6 Wireless Headphones", "displayedUrl": "sony.com", "prices": ["$399.99"] }
  ],
  "aiOverview": null,
  "organicResults": [
    {
      "title": "The Best Wireless Headphones for 2026",
      "url": "https://www.example-reviews.com/best-wireless-headphones",
      "displayedUrl": "example-reviews.com/best-wireless-headphones",
      "description": "We tested 40 pairs of wireless headphones over three months to find the best options for sound quality, noise cancelling, and battery life...",
      "emphasizedKeywords": ["best wireless headphones 2026"],
      "siteLinks": [],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    { "title": "best noise cancelling headphones 2026", "url": "https://www.google.com/search?q=best+noise+cancelling+headphones+2026&hl=en&gl=us", "type": "organic", "position": 1 }
  ],
  "peopleAlsoAsk": [
    { "answer": null, "question": "What are the top rated wireless headphones right now?", "title": "What are the top rated wireless headphones right now?", "url": null, "date": null }
  ],
  "customData": {
    "perplexitySearch": { "enablePerplexity": false, "searchRecency": null, "returnImages": false, "returnRelatedQuestions": false },
    "chatGptSearch": { "enableChatGpt": false },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": "https://api.apify.com/v2/key-value-stores/abcd1234efgh5678/records/best_wireless_headphones_2026_20260816_120000_p1.html"
}

html is added as an extra field only when saveHtml is enabled — it carries the full page source and can make the dataset large.

How does it work?

Google Search Results Scraper sends plain HTTP GET requests to google.com/search through Apify Proxy — starting with the GOOGLE_SERP proxy group and escalating to RESIDENTIAL proxies if Google's block/CAPTCHA page is detected. Each response HTML is parsed with CSS/XPath selectors tuned to Google's current SERP markup to pull out organic results, ads, shopping products, People Also Ask, and related searches. By default the primary request uses Google's stripped-down "Web" surface, so the Actor fires one extra request per query (page 1 only) against Google's plain surface to recover the real People Also Ask, related-searches, and sitelinks blocks that surface hides. Only data that's genuinely public — what any logged-out visitor sees — is returned; nothing is fabricated when a block is absent. The output schema stays the same fixed shape run to run, regardless of how Google reshuffles its page layout underneath.

Integrations

Google Search Results Scraper works wherever you already run automation — call it directly over HTTP, register it as an MCP tool for an AI agent, or wire it into a no-code workflow.

Calling Google Search Results Scraper programmatically

from apify_client import ApifyClient

client = ApifyClient("<APIFY_API_TOKEN>")
run = client.actor("scraper-engine/google-search-results-scraper").call(
    run_input={
        "queries": "best wireless headphones 2026",
        "maxPagesPerQuery": 3,
    }
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["organicResults"])

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request against the Apify API using your API token.

MCP integration for AI agents

Google Search Results Scraper is reachable through Apify's official MCP server. Register it with:

{
  "mcpServers": {
    "apify": {
      "command": "npx",
      "args": ["@apify/actors-mcp-server", "--actors", "scraper-engine/google-search-results-scraper"],
      "env": { "APIFY_TOKEN": "<APIFY_API_TOKEN>" }
    }
  }
}

This works with Claude Desktop, Claude Code, and any other MCP-compatible client.

No-code tools (n8n, Make, LangChain)

In n8n, use the Apify node (or an HTTP Request node against the run-sync endpoint) with your queries as input, and connect the output straight into downstream nodes. In Make, the Apify app's "Run an Actor" module accepts the same input JSON and returns the dataset items for the next module in your scenario. In LangChain, ApifyWrapper / ApifyDatasetLoader call the Actor and load its dataset items directly as documents for a retriever or agent tool.

Is it legal to scrape Google search results?

Yes — scraping publicly available Google search results is generally legal; these are search result pages, not personal data, so no personal-data regime like GDPR applies to them. Google Search Results Scraper only returns data any logged-out visitor can see on a public results page — it does not access personalized, logged-in, or account-gated results. Scraping search engine results still sits under Google's Terms of Service and, for the underlying pages you index, database-rights and copyright frameworks in some jurisdictions, so review your target use case against Google's ToS before large-scale or commercial reuse. Consult legal counsel if your use case involves bulk storage of personal data (e.g. names or contact details that happen to appear inside scraped result text).

Frequently asked questions

What Google search fields does Google Search Results Scraper return?

The top-level fields are organicResults, paidResults, paidProducts, peopleAlsoAsk, and relatedQueries, plus resultsTotal and an optional aiOverview. See What data can I extract above for the full field list.

Does Google Search Results Scraper require a Google account or login?

No. The Actor issues plain HTTP requests through Apify Proxy with no sign-in step — no Google account, cookies, or session is required to run it.

How many search results can I extract in one run?

queries accepts unlimited lines, and each query can pull as many pages as maxPagesPerQuery allows — there's no schema-enforced page cap. Google itself returns roughly 10 organic results per page regardless of resultsPerPage, so a maxPagesPerQuery of 10 targets around 100 organic results per query.

What happens if a query returns zero results or gets blocked?

If Google genuinely has nothing to show, the relevant array (organicResults, paidResults, etc.) simply comes back empty rather than fabricated. If every retry attempt for a page is blocked (proxy escalation exhausted), that page is skipped entirely — no dataset row is pushed for it, and nothing is charged, while a warning is logged for that query.

Can I scrape multiple Google searches at once?

Yes. Put one search term or full Google Search URL per line in queries — every line is scraped in the same run, in order, so there's no need for a separate call per keyword.

Does Google Search Results Scraper work with Claude, ChatGPT, and other AI agent tools?

Yes. It's reachable through Apify's official MCP server (npx @apify/actors-mcp-server), compatible with Claude Desktop, Claude Code, and other MCP clients — or callable as a plain HTTP endpoint by any agent framework.

How does Google Search Results Scraper compare to other Google SERP scrapers?

As observed on their Apify Store listings on 2026-08-16: Apify's own apify/google-search-scraper also extracts organic and paid results, AI Overview, and People Also Ask, and is priced at $1.80 per 1,000 scraped result pages. apidojo/google-search-scraper returns a narrower field set (title, link, visible_link, snippet, type) at $0.002 per query plus $0.0002 per additional result beyond the first 10. Google Search Results Scraper differentiates on the honesty of its PAA and related-searches extraction — both are read from Google's real blocks via a dedicated plain-surface request rather than synthesized from organic titles — and on returning shopping products (paidProducts) as a distinct array.

Does Google Search Results Scraper return data in a format LLMs can use directly?

Yes. Typed, normalized JSON with consistent field names across runs — pass it directly to an LLM, index it into a vector store, or feed it to an agent tool without writing an HTML parser.

What happens when Google changes its layout or anti-bot system?

The Actor is maintained and its output schema is designed to stay stable across Google layout changes; no specific turnaround time is promised for any given change.

Can I use Google Search Results Scraper without managing proxies or browser infrastructure?

Yes for proxies — Apify Proxy, group selection, and residential escalation on a detected block are all handled automatically; you never provision or rotate proxy IPs yourself. ⚠️ There is no managed browser: the Actor issues plain HTTP requests with no JavaScript execution, so anything Google only renders after a click (most PAA answer text, the generative AI Mode panel) is out of reach regardless of proxy setup.

Which Google search fields work best for AI training data and RAG indexing?

For RAG, index organicResults[].title, organicResults[].description, and organicResults[].url as retrievable passages, and peopleAlsoAsk[].question for FAQ-style grounding. For training or feature data, resultsTotal, organicResults[].position, and searchQuery.countryCode are the most consistently structured fields across every row — all fields return as typed primitives, arrays, or objects, never as raw HTML.

Related scrapers

Scraper	What it extracts
Google Search Results (SERP) Scraper	A lighter, quick-setup Google SERP scraper on the same account
Google Scholar Scraper	Academic search results, authors, and citation counts
Google Maps Reviews Scraper	Reviews and ratings from Google Maps business listings
Google Play Scraper	App listings, ratings, and reviews from the Google Play Store
DuckDuckGo Scraper	SERP data from DuckDuckGo, an alternative search engine
Baidu Search Scraper	SERP data from Baidu, China's leading search engine

Your feedback

Found a bug or a field that doesn't match what Google actually returns? Open an issue on this Actor's Issues tab in Apify Console, or reach out through the Scraper Engine Store listing — bug reports and missing-field requests are how this Actor stays accurate as Google's SERP markup changes.