Google Search Results (Serp) Scraper

Google SERP Scraper — Organic Results, Ads and People Also Ask

Google Search Results (SERP) Scraper runs a live Google search for any keyword and returns organic results, paid ads, shopping products, People Also Ask questions, related queries, and an optional AI Overview as structured JSON — no HTML parsing required. Enter a keyword, hit Start, and the dataset fills in with one normalized row per query.

What is Google Search Results (SERP) Scraper?

Google Search Results (SERP) Scraper is an Apify Actor that fetches google.com/search for one or more keywords and parses the response page into a consistent JSON schema — organic listings, ads, shopping cards, People Also Ask, related searches, and result counts. No Google account, login, or API key is required to run it; it queries Google anonymously through Apify's proxy network. It's built for SEO teams tracking rankings, marketers doing keyword and ad research, and developers or AI engineers who need machine-readable SERP data for pipelines and RAG systems.

What Google Search result data is publicly available to scrape?

Everything a signed-out visitor sees on a Google results page is publicly accessible without logging in — organic listings, ads, shopping cards, and the People Also Ask box. What isn't accessible from an anonymous request is anything Google renders based on a signed-in session.

Data category	Publicly available (anonymous search)	Restricted (signed-in Google account)
Organic listings (title, URL, snippet)	Yes	—
Paid ads and shopping product cards	Yes	—
People Also Ask questions	Yes	—
Related searches	Yes	—
Estimated total result count	Yes	—
Personalized ranking (search history, browsing history)	No	Requires a signed-in Google session
Google AI Overview	Only via a separate SERP API key passed in input	Rolled out per account/region by Google

Google Search Results (SERP) Scraper only returns publicly visible data — what any visitor sees. Nothing behind a login wall.

What data can I extract with Google Search Results (SERP) Scraper?

Every run pushes one JSON object per keyword/page combination to the dataset, built from these keys:

Query and page identity fields

Field	Description
searchQuery	Object with term, url, device ("DESKTOP"), page, type ("SEARCH"), domain ("google.com"), countryCode, languageCode, locationUule, resultsPerPage
searchQueryTerm	The final query string actually sent to Google (after any query modifiers are applied)
url	The live Google search results URL that was fetched
hasNextPage	true if Google's results page exposes a next-page link and more pages remain to fetch
serpProviderCode	Fixed provider code, always "O"
resultsTotal	Google's estimated total result count, parsed from the "About X results" text on the page

SERP result blocks

Field	Description
organicResults	Array of organic listings: title, url, displayedUrl, description, emphasizedKeywords, siteLinks, productInfo, type ("organic"), position, and icon when icon extraction is enabled
paidResults	Array of text ads with the same shape as organicResults but type: "paid"; description and siteLinks are not populated for ads
paidProducts	Array of shopping/product cards: title, displayedUrl, prices (list of price strings)
peopleAlsoAsk	Array of { question, title, url, answer, date } — answer and date are always null; the parser captures the question text and its link, not the expanded answer
relatedQueries	Array of { title, url } built from the top 10 organic result titles, reformatted as Google search links — not Google's native "related searches" box
suggestedResults	Same items as relatedQueries, reshaped as { title, url, type: "organic", position }
aiOverview	{ text, references } or null. Only populated when Google AI Mode is enabled and a third-party SERP API key is supplied — both are advanced fields passed via raw JSON input, not exposed in the input form. On a default run this is always null
customData	{ perplexitySearch, chatGptSearch, maximumLeadsEnrichmentRecords, leadsEnrichmentDepartments } — reflects optional add-on toggles that, like aiOverview, are only reachable via raw JSON input and default to off/0
htmlSnapshotUrl	Key-value store key for the saved raw HTML page, or null if HTML saving wasn't enabled
html	Full raw HTML of the fetched page(s) — present only when HTML saving is enabled

⚠️ aiOverview is the field most likely to surprise you: it stays null unless you pass aiMode and serpApiKey directly in the run's JSON input (there's no form field for either), so don't rely on it for AI Overview data on a standard run.

🤖 Add-on: Need additional Google data?

Pair this Actor with Google Maps Reviews Scraper for review data on local businesses, Google Scholar Scraper for academic search results, or Google Play Scraper for app-store listings — all published under the same Scraper Engine account.

How does Google Search Results (SERP) Scraper differ from Google's official Search API?

Google's own Custom Search JSON API queries a Programmable Search Engine index you configure in advance, not a live rendering of google.com/search — so it has no equivalent for ads, shopping cards, or People Also Ask. As of the official pricing page (checked 2026-08-16), the API is also closed to new customers and scheduled for discontinuation on January 1, 2027.

Feature	Custom Search JSON API	Google Search Results (SERP) Scraper
New sign-ups	Closed to new customers	Open — run directly on Apify, no approval step
Free quota	100 queries/day free	No Google-imposed query quota
Paid pricing	$5 per 1,000 queries, up to 10,000/day	Apify pay-per-result pricing (see Pay-per-event below)
Setup	Requires creating and configuring a Programmable Search Engine first	Enter a keyword and run
Ads / shopping data	Not returned	paidResults and paidProducts returned directly
People Also Ask / related searches	Not returned	peopleAlsoAsk and relatedQueries returned
Lifecycle	Discontinuing January 1, 2027	No comparable end date

Use the Custom Search JSON API if you're an existing customer with an already-configured search engine and only need indexed web results. Use Google Search Results (SERP) Scraper when you need the actual rendered SERP — ads, shopping cards, and People Also Ask included — without a Google-side setup step.

How to use Google Search Results (SERP) Scraper

Google Search Results (SERP) Scraper runs entirely inside the Apify platform — there's no separate signup or API key required to start a run.

1. Open the Actor on its Apify Store page and go to the Input tab.
2. Enter your search query in Search Keyword(s) (keyword) — the only required field.
3. Optionally set Number of results (limit), Country (country), and language/date filters.
4. Click Start.
5. Open the Output/Dataset tab and export the results as JSON or CSV (or any of Apify's other dataset export formats).

How to scale to bulk search query extraction

keyword is a multi-line textarea — put one query per line and every line is processed as a separate search within the same run. You can also paste a full Google search results URL on a line and the Actor extracts the q= parameter automatically. There's no separate array-input field; bulk scraping is simply multiple lines in one keyword value.

What can you do with Google SERP data?

• 📈 An SEO analyst tracking rankings uses organicResults[].position and organicResults[].url to monitor where target domains sit for a keyword over time.
• 📝 A content strategist building FAQ pages uses peopleAlsoAsk[].question to source real search-intent questions worth answering.
• 💰 A PPC analyst auditing competitor ad copy uses paidResults[].title and paidResults[].displayedUrl to see which advertisers are bidding on a keyword.
• 🛒 A pricing analyst benchmarks shopping listings using paidProducts[].title and paidProducts[].prices pulled from the shopping carousel.
• 🤖 An AI engineer builds a RAG pipeline that indexes organicResults[].description and peopleAlsoAsk[].question as grounding context so an agent can answer "what does Google currently rank for X" questions with cited sources.

How does Google Search Results (SERP) Scraper handle rate limits and blocking?

Every request runs through Apify's Proxy — by default the Actor requests through the GOOGLE_SERP proxy group, with proxy_location (us/ca) forcing the proxy region when set. Each page fetch retries up to 3 times on a non-200 response or a 429. If Google's block page is detected in the response ("unusual traffic" / "trouble accessing Google"), the Actor automatically escalates to Apify's RESIDENTIAL proxy group, retries up to 3 more times, and then keeps using residential proxies for the rest of that keyword's run. If a query still can't be fetched after all retries, the Actor logs a warning and skips it — it does not push a partial or fabricated row for a keyword it couldn't retrieve.

⬇️ Input

keyword is the only required field. Everything else is optional and controls pagination, localization, and proxy region.

Parameter	Required	Type	Default	Description
keyword	Yes	string	"nike"	One full search query per line — each line is a separate search. Full Google search URLs are also accepted.
include_merged	No	boolean	true	Adds one extra dataset row per keyword that merges results from all fetched pages (searchQuery.page = "all").
limit	No	string (enum: 10, 20, 30, 40, 50, 100, all)	"all"	Target result count. Each page holds up to 10 results; "all" fetches as many pages as possible up to a safe internal limit.
start	No	integer	—	Offset for the first result index (Google's start parameter).
page	No	integer	—	Fetch only this specific result page (1 = first page). When set, the run is limited to that single page.
country	No	string (enum, ISO 3166-1 alpha-2)	—	Main country selector — sets Google's gl parameter and the default proxy region.
gl	No	string (enum, ISO 3166-1 alpha-2)	—	Overrides the local country for results (Google's gl parameter) independently of country.
hl	No	string (enum)	—	Language of the Google Search interface (menus, buttons), e.g. en, fr, de.
tbs	No	string	—	Advanced time filter, e.g. qdr:d (past day), qdr:w (past week), or a custom range like cdr:1,cd_min:01/01/2024,cd_max:01/31/2024.
lr	No	string (enum, e.g. lang_en)	—	Restricts results by content language (Google's lr parameter).
cr	No	string (enum, e.g. countryUS)	—	Restricts results by country of origin (Google's cr parameter).
proxy_location	No	string (enum: us, ca)	—	Forces the proxy region independently of the Google country used for results.

Example input

{
  "keyword": "best running shoes 2026\nnike air max",
  "include_merged": true,
  "limit": "50",
  "start": 0,
  "country": "US",
  "gl": "US",
  "hl": "en",
  "tbs": "qdr:w",
  "lr": "lang_en",
  "cr": "countryUS",
  "proxy_location": "us"
}

Leave page out (as above) to let limit drive pagination automatically — setting page forces the run to fetch only that one page regardless of limit.

⬆️ Output

Each dataset row is typed, normalized JSON with a stable key set across runs, regardless of query or Google's current page markup. Export runs as JSON, CSV, Excel, HTML table, or RSS directly from the Apify dataset UI, or pull them via the API.

Example output

{
  "searchQuery": {
    "term": "nike air max",
    "url": "https://www.google.com/search?q=nike+air+max&gl=us&hl=en",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "nike air max",
  "url": "https://www.google.com/search?q=nike+air+max&gl=us&hl=en",
  "hasNextPage": true,
  "serpProviderCode": "O",
  "resultsTotal": 458000000,
  "relatedQueries": [
    { "title": "Nike Air Max 2026 - Official Nike Site", "url": "https://www.google.com/search?q=Nike+Air+Max+2026+-+Official+Nike+Site&hl=en&gl=us" }
  ],
  "paidResults": [
    { "title": "Nike Air Max | Official Nike Store", "url": "https://www.nike.com/air-max", "displayedUrl": "www.nike.com/air-max", "description": "", "emphasizedKeywords": ["nike air max"], "siteLinks": [], "productInfo": {}, "type": "paid", "position": 1 }
  ],
  "paidProducts": [
    { "title": "Nike Air Max 90", "displayedUrl": "www.nike.com", "prices": ["$130.00"] }
  ],
  "aiOverview": null,
  "organicResults": [
    {
      "title": "Nike Air Max. Nike.com",
      "url": "https://www.nike.com/air-max",
      "displayedUrl": "www.nike.com/air-max",
      "description": "Shop the latest Nike Air Max shoes, from the Air Max 90 to the Air Max Plus.",
      "emphasizedKeywords": ["nike air max"],
      "siteLinks": [
        { "title": "Air Max 90", "url": "https://www.nike.com/air-max-90", "description": "" }
      ],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    { "title": "Nike Air Max 2026 - Official Nike Site", "url": "https://www.google.com/search?q=Nike+Air+Max+2026+-+Official+Nike+Site&hl=en&gl=us", "type": "organic", "position": 1 }
  ],
  "peopleAlsoAsk": [
    { "answer": null, "question": "What is the most popular Nike Air Max?", "title": "What is the most popular Nike Air Max?", "url": "https://www.google.com/search?q=most+popular+nike+air+max", "date": null }
  ],
  "customData": {
    "perplexitySearch": { "enablePerplexity": false, "searchRecency": null, "returnImages": false, "returnRelatedQuestions": false },
    "chatGptSearch": { "enableChatGpt": false },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": null
}

How does it work?

Google Search Results (SERP) Scraper sends each query to google.com/search through Apify's Proxy, starting with the GOOGLE_SERP proxy group and escalating to residential proxies automatically if Google's block page is detected. The returned HTML is parsed with CSS selectors covering multiple known Google markup variants (organic result containers, ad blocks, shopping cards, and the People Also Ask box), with a fallback pass that scans generic heading links if fewer than 5 organic results are found. Only what's visible to a signed-out visitor is returned — nothing requires a Google login. Because parsing targets several selector variants rather than one fixed layout, the output schema stays the same shape even as Google tweaks its page markup.

Integrations

Google Search Results (SERP) Scraper runs on Apify, so it works with anything that can call the Apify API or an Apify-compatible integration.

Calling Google Search Results (SERP) Scraper programmatically

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_API_TOKEN>")

run_input = {
    "keyword": "nike air max",
    "limit": "50",
    "country": "US",
}

run = client.actor("google-search-results-serp-scraper").call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["organicResults"])

Works in Go, Ruby, Node.js, cURL — any language that can make an HTTP request to the Apify API.

MCP integration for AI agents

The Actor is reachable through Apify's hosted MCP server. Point an MCP-compatible client (Claude, Cursor, or any MCP client) at:

https://mcp.apify.com?tools=<your-apify-username>/google-search-results-serp-scraper

or install it via the Apify CLI: apify mcp install cursor --tools <your-apify-username>/google-search-results-serp-scraper.

No-code tools (n8n, Make, LangChain)

In n8n, use the HTTP Request node (or the Apify node) pointed at the Actor's run endpoint with your API token. In Make, use the Apify app's "Run an Actor" module and map keyword from an upstream module. In LangChain, the langchain-apify integration package wraps any Apify Actor — including this one — as a callable tool for an agent chain.

Is it legal to scrape Google search results?

Scraping publicly available Google search result pages is generally legal — search results are public web content, not personal data, so no personal-data regulation like GDPR applies to the dataset itself. Google Search Results (SERP) Scraper only returns what a signed-out visitor can already see on a results page. What does apply is Google's own Terms of Service, which restrict automated querying of google.com, and database-rights or copyright considerations if you plan to republish scraped content wholesale rather than use it for analysis. Consult legal counsel if your use case involves bulk storage of personal data (for example, names that appear inside snippets or People Also Ask text).

Frequently asked questions

What Google SERP fields does Google Search Results (SERP) Scraper return?

The top fields are organicResults, paidResults, paidProducts, peopleAlsoAsk, and relatedQueries — see What data can I extract above for the full field list.

Does Google Search Results (SERP) Scraper require a Google account or login?

No. It queries Google anonymously through Apify's proxy network — no Google account, cookies, or login session are used or required.

How many search results can I extract in one run?

That depends on limit: each Google results page holds up to 10 results, so limit: "50" fetches 5 pages per keyword, and limit: "all" (the default) fetches up to 100 pages per keyword before stopping, or fewer if Google's next-page link runs out first.

What happens if a query returns zero results?

The Actor still pushes a dataset row for that keyword — organicResults, paidResults, and peopleAlsoAsk come back as empty arrays and resultsTotal is 0. This is different from a blocked request, which is retried and, if it never succeeds, is skipped with a logged warning and no row at all.

Can I run multiple Google searches in one call?

Yes. Put one query per line in keyword — every line is scraped as a separate search within the same run and produces its own dataset rows.

Does Google Search Results (SERP) Scraper work with Claude, ChatGPT, and other AI agent tools?

Yes. It's reachable through Apify's hosted MCP server (see Integrations above) for MCP-native clients, and callable as a plain HTTP endpoint by any agent framework that can call the Apify API.

How does Google Search Results (SERP) Scraper compare to Google's own Search API?

Unlike the Custom Search JSON API, this Actor needs no Programmable Search Engine setup and returns ads, shopping cards, and People Also Ask alongside organic results in one JSON row — see the comparison table above for the full breakdown.

Does Google Search Results (SERP) Scraper return data in a format LLMs can use directly?

Yes. Output is typed, normalized JSON with consistent field names across runs — no HTML parsing or CSS selectors needed. Pass it directly to an LLM, index it into a vector store, or feed it to an agent tool.

What happens when Google changes its layout or anti-bot system?

The parser targets multiple known selector variants for each result type rather than one fixed layout, and falls back to a generic link-scan if fewer than 5 organic results are found, so the output schema is designed to stay stable across minor markup changes.

Can I use Google Search Results (SERP) Scraper without managing proxies or browser infrastructure?

Yes. Proxy selection, rotation, and escalation from Apify's GOOGLE_SERP proxy group to RESIDENTIAL proxies on a block are all handled internally — you don't configure or pay for proxies separately.

Which Google SERP fields work best for AI training data and RAG indexing?

For RAG grounding, organicResults[].description, organicResults[].title, and peopleAlsoAsk[].question carry the most usable free text. For structured training data, organicResults[].position, resultsTotal, and paidProducts[].prices are the most consistently typed across records. All fields return as typed primitives (strings, integers, booleans, or nested objects/arrays) — no manual parsing required.

Related scrapers

Scraper	What it extracts
Google Search Results Scraper	An alternate Google SERP scraper in the Scraper Engine catalog
Google Scholar Scraper	Academic search results from Google Scholar
Google Maps Reviews Scraper	Reviews and ratings for local businesses on Google Maps
Google Play Scraper	App listings and metadata from the Google Play Store
DuckDuckGo Scraper	Search results from DuckDuckGo
Baidu Search Scraper	Search results from Baidu

Your feedback

Found a bug or missing a field in Google Search Results (SERP) Scraper? Let us know through the Actor's issue tracker on its Apify Store page — bug reports and feature requests are what keep the parser matched to Google's current markup.