Google Search Results (SERP) Scraper & Shopping Prices

Google Search Results Scraper — SERP and Merchant Shopping Prices

This Actor scrapes Google's organic search results and Google Shopping in a single pass, returning organicResults, paidResults, peopleAlsoAsk and relatedQueries for every keyword, plus a shoppingResults[] array of real offers — merchant, priceText, originalPriceText, discountPercent, rating and reviewsText — attached to the same row. Price-monitoring teams, e-commerce SEOs and market researchers use it to see organic ranking and shopping presence for one keyword side by side, instead of joining two separate datasets by hand. Feed it a list of product keywords and each run comes back as one typed JSON row per query, ready for a price tracker or a keyword-intent dashboard.

---

What is Google Search Results (SERP) Scraper & Shopping Prices?

It is a keyword-in, JSON-out scraper of google.com/search: give it one or more keywords (or full Google Search URLs) and it returns the organic SERP for each one — titles, links, descriptions, ads, People Also Ask, related searches — and, on the same row, the priced offers Google's Shopping surface (udm=28) shows for that keyword. No Google account, API key, or Custom Search Engine setup is required; the Actor runs on the Apify platform and manages its own Apify Proxy sessions, escalating from the Google SERP proxy group to residential IPs when a request gets blocked.

Key features:

• 🔎 Keyword or URL input — plain keywords or full Google Search URLs, one per line; product-shaped keywords return the richest shoppingResults[]
• 🛒 Shopping offers on every row — merchant, price as printed, "was" price, discount badge, delivery text, star rating and review count, fetched once per keyword and attached to every page row plus the merged row
• 🏷️ Condition and sponsorship filters — restrict offers to new / used / refurbished via Google's own filter chip, and keep or drop sponsored cards
• 🌍 Full localization — market country, results country (gl), interface language (hl), content language (lr), country of origin (cr) and a raw Google time filter (tbs)
• 🎯 Commercial-intent triage — onlyKeywordsWithShopping skips, and does not charge for, any keyword whose Shopping surface returned zero offers
• 📤 Standard Apify dataset output — JSON, CSV, Excel, or any other format the Apify dataset export supports, plus named views for organic results, paid results, and shopping offers specifically

---

What data can I extract with Google Search Results (SERP) Scraper & Shopping Prices?

Every keyword produces one SERP row (and, if include_merged is on, one extra merged row) carrying the fields below — every key the Actor's row-building code writes, not just the dataset's default view columns.

Field	Example Value	Use Case
searchQuery	{"term":"nike air max 90","page":1,"countryCode":"US",...}	Trace which literal query, page and locale produced this row
searchQueryTerm	"nike air max 90"	Flat keyword column for grouping and filtering
url	"https://www.google.com/search?q=..."	Re-open the exact SERP that was scraped
hasNextPage	true	Know whether more organic pages exist past this row
serpProviderCode	"O"	Provider marker carried over from the base contract
resultsTotal	1980000000	Google's own estimate of total matching pages
relatedQueries	[{"title":"nike air max 90 white","url":"..."}] or null	Keyword expansion from Google's real related-searches block
paidResults	[{"title":"Nike Official Site","url":"...","position":1}]	See which advertisers bid on the keyword
paidProducts	[{"title":"Nike Air Max 90","displayedUrl":"nike.com","prices":["$130"]}]	Catch Shopping ad tiles embedded directly in the organic SERP
aiOverview	null	Google's AI Overview text and source references, when served
organicResults	[{"title":"Nike Air Max 90 Shoes","url":"https://www.nike.com/...","position":1}]	The organic ranking itself — the core SEO use case
suggestedResults	[{"title":"nike air max 90 white","url":"...","type":"relatedSearch"}] or null	Related queries reshaped as a result list
peopleAlsoAsk	[{"question":"Are Nike Air Max 90 true to size?","url":"..."}]	Content and FAQ research
customData	{"perplexitySearch":{...},"chatGptSearch":{...},"maximumLeadsEnrichmentRecords":0}	Reserved enrichment block, left at its defaults by this input schema
htmlSnapshotUrl	null	Key-value-store pointer to the raw HTML page, only set when a snapshot was saved
shoppingResults	[ … offer objects, see below … ]	The priced offers for this keyword
hasShoppingResults	true	One-column filter for "does this keyword sell anything"
shoppingResultsCount	40	How many offers were collected for the row
shoppingSurfaceUrl	"https://www.google.com/search?q=...&udm=28"	The exact Shopping URL that was fetched
shoppingSelectorRung	"I8iMf"	Which CSS selector rung matched the offer cards — a debugging aid against Google's class-name rotation
shoppingConditionRequested	"new"	Echoes the shoppingCondition input
shoppingConditionApplied	null	Confirms whether Google actually offered and applied that condition chip
shoppingPagesFetched	1	How many &start= pages were walked to collect offers
shoppingError	null	Why offers came back empty, when they did (e.g. SHOPPING_BLOCKED_OR_NO_CARDS)
isCharged	true	Whether this row consumed a row_result billing event
errorReason	null	Populated only on the uncharged accounting row (see Pay-per-event below)
errorDetail	null	Extra context for errorReason

🛒 Shopping offer fields

Each entry in shoppingResults[] is parsed straight off a Google Shopping offer card:

Field	Example Value
position	1
title	"Nike Men's Air Max 90"
merchant	"footdistrict.com"
priceText	"$138.41"
price	138.41
currency	"$"
originalPriceText	"$150.00" or null
discountPercent	null, or a percent-off string exactly as Google's badge prints it
deliveryText	"Free delivery"
rating	4.6
reviewsText	"14K"
isSponsored	false
googleProductViewerUrl	"https://www.google.com/search?q=...&udm=28#pvs=0&rank=1"
searchKeyword	"nike air max 90"

⚠️ No per-offer ID is shipped. Google prints no stable identifier anywhere inside an offer card on the HTML surface this Actor reads, so shoppingResults[] carries no offerId. De-duplication and run-to-run change tracking key on title + merchant + priceText instead — if a merchant changes its price, the next run's offer will look like a new line rather than an updated one, so compare by title+merchant, not by array position. A field Google did not print (returns wording, per-offer condition) is left null rather than guessed — nothing is back-filled from another field.

🎯 Targeting and localization fields

country (and its overrides gl, hl, lr, cr) decide which Google market is queried, and — for shopping — which merchants and which currency come back: the same keyword in us and de returns a different offer set. shoppingCondition and includeSponsoredOffers narrow the offer list itself; isSponsored on every offer lets you re-filter after the fact even if you kept sponsored cards. proxy_location (us or ca) pins the proxy exit independently of the Google market when you need the request to originate from a specific region.

---

How does Google Search Results (SERP) Scraper & Shopping Prices differ from the official Google Custom Search API?

Google's own programmatic option, the Custom Search JSON API (Programmable Search Engine), is closed to new customers and is being discontinued for existing ones on January 1, 2027, so it is not a viable path for a new bulk-scraping project at all. Even for existing customers it only searches the one Programmable Search Engine instance you configure in the Control Panel — not the live open web — and Google's own documentation states it "returns up to the first 100 results only" per query. It also has no shopping or price data in its response shape. (Checked against developers.google.com/custom-search/v1 on 2026-08-16.)

Feature	Google Custom Search JSON API	Google Search Results (SERP) Scraper & Shopping Prices
New signups	Closed to new customers; discontinuing Jan 1, 2027	Open now — run it from the Apify Console today
Search scope	Only the search engine instance you index yourself	The live public google.com SERP, for any keyword
Result cap	Up to the first 100 results, per Google's docs	limit walks up to all organic pages Google paginates; shopping offers capped independently at 0–200
Shopping / price data	Not part of the response	shoppingResults[] — merchant, price, was-price, discount, delivery, rating
Setup	Requires a Control Panel search engine (cx) plus an API key	One keyword field; no account or API key
Pricing	Free: 100 queries/day. Paid: $5 per 1,000 queries, up to 10,000/day	Apify pay-per-event, billed on the row_result event

Why can't I just use the Custom Search JSON API for this?

Because it isn't searching Google at all in the way most people expect — it searches your own configured search engine index, which you have to build and maintain in the Control Panel first. That makes it unsuitable for ad-hoc keyword or shopping research on arbitrary products. And since Google closed it to new sign-ups and set a 2027 shutdown date for existing ones, it is not a foundation to build a new pipeline on regardless of scope. This Actor instead requests the same public SERP and Shopping pages any browser sees, so it works on any keyword from the first run, with organic results and shopping offers merged into one row.

Use the Custom Search JSON API only if you're an existing customer with an already-configured search engine limited to your own site set. Use this Actor when you need results from the open web, shopping prices, or simply don't have — and don't want to build — a Custom Search Engine.

---

How to use data extracted from Google Search results?

💰 Price monitoring and e-commerce teams

Feed a product keyword list with includeShoppingResults: true and a maxShoppingResultsPerQuery high enough to see the market (up to 200). Re-run on a schedule and diff priceText and merchant per title+merchant key to catch price drops, new sellers, or your own listing falling out of the offer set. shoppingCondition: "new" and includeSponsoredOffers: false keep the comparison to organic, new-condition competitors only.

🏢 Agencies and SEO/PPC service providers

Run the same keyword set for multiple clients and pull organicResults (ranking and description) alongside paidResults (who's bidding) in one dataset, instead of two separate tools. country/gl/hl let one client's multi-market audit — e.g. us, gb, de — run as three inputs against the identical keyword list, with results directly comparable row to row.

📊 Market research and intelligence

Compare shoppingResultsCount and the distinct merchant set across country codes for the same keyword to see where a product category has real retail depth versus none. Pairing hasShoppingResults with onlyKeywordsWithShopping turns a long, mixed keyword list into a clean split between commercial-intent and informational-intent terms without manual review.

🤖 AI agents and automated pipelines

Because the output is one typed JSON object per keyword, it drops straight into an agent tool call or a RAG index: an agent checking "what does this cost right now" calls the Actor, reads shoppingResults[0].priceText, and answers without a human in the loop. See MCP for AI agents below for the exact registration.

---

🔼 Input sample

keyword is the only required field. Everything else has a schema default or is left unset (meaning: use Google's own default for that parameter).

Parameter	Required	Type	Description	Example Value
keyword	Yes	string (textarea)	One search per line — plain keywords or full Google Search URLs. Default "nike".	"nike air max 90"
include_merged	No	boolean	Add one extra row per keyword combining all pages (searchQuery.page = "all"). Default true.	true
includeShoppingResults	No	boolean	Fetch Google's Shopping surface once per keyword. Default true.	true
maxShoppingResultsPerQuery	No	integer (0–200)	Upper bound on shoppingResults[] per keyword. Default 40.	100
shoppingCondition	No	string, enum: any, new, used, refurbished	Restrict offers to a condition chip Google offers on the page. Default "any".	"new"
includeSponsoredOffers	No	boolean	Keep offer cards Google marks Sponsored. Default true.	false
onlyKeywordsWithShopping	No	boolean	Skip, and do not charge for, keywords with zero offers. Default false.	true
limit	No	string, enum: 10,20,30,40,50,100,all	How many organic result pages to walk (~10 results/page). Default "all".	"50"
start	No	integer	Google start offset for the first organic page. Wins over page. No default.	20
page	No	integer	Fetch only this one organic page. Ignored if start is set. No default.	2
deduplicateAcrossPages	No	boolean	Off = page n re-contains pages 1..n (original shape). Default false.	false
country	No	string, enum (ISO 3166-1 alpha-2)	Primary market — sets Google gl and the default proxy region. No default.	"us"
gl	No	string, enum (ISO 3166-1 alpha-2)	Override the results country (gl). No default.	"de"
hl	No	string, enum (language codes, e.g. en, fr, zh-CN)	Interface language. No default.	"en"
lr	No	string, enum (lang_ + code)	Restrict content language. No default.	"lang_en"
cr	No	string, enum (country + code)	Restrict country of origin. No default.	"countryUS"
tbs	No	string (free text)	Raw Google time filter, e.g. qdr:d. No default.	"qdr:w"
proxy_location	No	string, enum: us, ca	Pin the proxy exit region independently of the Google market. No default.	"us"
fetchFullSerpPage	No	boolean	Off = original udm=14 request; on = real relatedQueries block. Default false.	false

{
  "keyword": "nike air max 90\nsony wh-1000xm5",
  "includeShoppingResults": true,
  "maxShoppingResultsPerQuery": 80,
  "shoppingCondition": "any",
  "includeSponsoredOffers": true,
  "onlyKeywordsWithShopping": false,
  "limit": "10",
  "country": "us",
  "hl": "en",
  "include_merged": true
}

Common pitfall: limit and maxShoppingResultsPerQuery are independent knobs. limit only controls how many organic pages are walked — it does nothing to the shopping offer count, which is bounded separately by maxShoppingResultsPerQuery. Setting limit: "10" will not shrink shoppingResults[], and raising maxShoppingResultsPerQuery will not fetch more organic pages. Also note that shoppingCondition is a best-effort filter chip, not a guaranteed URL parameter — check shoppingConditionApplied on the row rather than assuming the requested condition was honored.

---

🔽 Output sample

Output is typed, normalized JSON in the Apify dataset, exportable as JSON, CSV, Excel, XML, RSS or HTML table like any Apify dataset. The Actor also ships three named dataset views — organicResults, paidResults, and shoppingResults — for pulling just one slice without hand-picking columns.

{
  "searchQuery": {
    "term": "nike air max 90",
    "url": "https://www.google.com/search?q=nike+air+max+90&gl=us&hl=en",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "nike air max 90",
  "url": "https://www.google.com/search?q=nike+air+max+90&gl=us&hl=en",
  "hasNextPage": true,
  "serpProviderCode": "O",
  "resultsTotal": 198000000,
  "relatedQueries": null,
  "paidResults": [],
  "paidProducts": [],
  "aiOverview": null,
  "organicResults": [
    {
      "title": "Nike Air Max 90 Shoes",
      "url": "https://www.nike.com/w/air-max-90-shoes",
      "displayedUrl": "www.nike.com/w/air-max-90-shoes",
      "description": "Shop Nike Air Max 90 shoes in classic and new colorways.",
      "emphasizedKeywords": ["nike air max 90"],
      "siteLinks": [],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": null,
  "peopleAlsoAsk": [
    {
      "answer": null,
      "question": "Are Nike Air Max 90 true to size?",
      "title": "Are Nike Air Max 90 true to size?",
      "url": "https://www.google.com/search?q=...",
      "date": null
    }
  ],
  "customData": {
    "perplexitySearch": { "enablePerplexity": false, "searchRecency": null, "returnImages": false, "returnRelatedQuestions": false },
    "chatGptSearch": { "enableChatGpt": false },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": null,
  "shoppingResults": [
    {
      "position": 1,
      "title": "Nike Men's Air Max 90",
      "merchant": "footdistrict.com",
      "priceText": "$138.41",
      "price": 138.41,
      "currency": "$",
      "originalPriceText": "$150.00",
      "discountPercent": null,
      "deliveryText": "Free delivery",
      "rating": 4.6,
      "reviewsText": "14K",
      "isSponsored": false,
      "googleProductViewerUrl": "https://www.google.com/search?q=nike+air+max+90&udm=28#pvs=0&rank=1",
      "searchKeyword": "nike air max 90"
    }
  ],
  "hasShoppingResults": true,
  "shoppingResultsCount": 40,
  "shoppingSurfaceUrl": "https://www.google.com/search?q=nike+air+max+90&udm=28&gl=us&hl=en",
  "shoppingSelectorRung": "I8iMf",
  "shoppingConditionRequested": "any",
  "shoppingConditionApplied": null,
  "shoppingPagesFetched": 1,
  "shoppingError": null,
  "isCharged": true,
  "errorReason": null,
  "errorDetail": null
}

Pay-per-event billing. The Actor charges the row_result event on every row it pushes with isCharged: true. Two situations do not cost anything: a keyword that fails after retries (blocked or unparseable) is still pushed as an uncharged accounting row with isCharged: false and errorReason set, so failures are visible in the dataset without being billed; and with onlyKeywordsWithShopping: true, a keyword whose Shopping surface returned no offers is skipped entirely — not pushed, not charged. To exclude both cases when reading the dataset, filter on isCharged === true (equivalently, errorReason === null).

---

How do you filter and target specific search results?

Four axes control exactly what comes back. Keyword vs. URL: keyword accepts either plain text or a full Google Search URL — pass a URL when you already have tbs/gl/hl baked into it from a browser session; otherwise plain keywords are cleaner and combine with the dedicated locale fields. Scope precision: country/gl pick the market (and, for shopping, the merchant set and currency), hl/lr/cr narrow interface language, content language and country of origin independently, and tbs applies a raw Google time filter such as qdr:w. Quality thresholds: shoppingCondition restricts offers to new/used/refurbished via Google's own filter chip (confirm with shoppingConditionApplied), and includeSponsoredOffers plus the per-offer isSponsored flag control and expose sponsorship. Volume controls: limit (organic pages, up to all) and maxShoppingResultsPerQuery (0–200 offers) are independent caps — set each for what you actually need, since raising one does not raise the other.

{ "keyword": "nike air max 90", "shoppingCondition": "new", "includeSponsoredOffers": false, "country": "us" }

{ "keyword": "best running shoes\nnike air zoom pegasus", "onlyKeywordsWithShopping": true, "maxShoppingResultsPerQuery": 20 }

{ "keyword": "site:nike.com air max", "includeShoppingResults": false, "limit": "50", "country": "de", "hl": "de" }

---

▶️ Want to try other Google scrapers?

Scraper Name	What it extracts
Google Search Results Scraper — question and keyword discovery	Organic SERP, People Also Ask and keyword-discovery data without the shopping add-on
Google Maps Reviews Scraper — bad review finder	Google Maps reviews, filtered for negative-review triage
Google Play Scraper — country pricing and in-app purchases	App Store listing data, country-level pricing and in-app purchase info
Google Scholar Scraper — institution and journal data	Academic search results with institution and journal metadata
Amazon Price Tracker — title, image and product updates	Amazon product price and listing-change tracking
eBay Product Listing Scraper — with sold-price data	eBay listings including sold-price history
Alibaba Listings Scraper — and supplier profiles	Alibaba product listings plus supplier profile data
Amazon Search Scraper — by country and currency	Amazon search results localized by country and currency

---

How to extract Google data programmatically

The Actor runs as a standard Apify Actor: one API call starts a run, and the finished dataset is one JSON array, one object per keyword row, ready to page or stream.

🐍 Python example

from apify_client import ApifyClient

client = ApifyClient("<APIFY_API_TOKEN>")

run_input = {
    "keyword": "nike air max 90\nsony wh-1000xm5",
    "country": "us",
    "shoppingCondition": "new",
    "maxShoppingResultsPerQuery": 100,
}

run = client.actor(
    "captivated_transformer/google-search-results-serp-scraper-and-shopping-prices"
).call(run_input=run_input)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["searchQueryTerm"], item["shoppingResultsCount"])
    for offer in item.get("shoppingResults", [])[:3]:
        print(" ", offer["merchant"], offer["priceText"])

🔌 MCP for AI agents

The Actor is reachable through Apify's MCP server, so any MCP-compatible client can call it as a tool without custom glue code:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=captivated_transformer/google-search-results-serp-scraper-and-shopping-prices",
      "headers": { "Authorization": "Bearer <APIFY_TOKEN>" }
    }
  }
}

Compatible clients include Claude Desktop, Cursor, VS Code (GitHub Copilot agent mode), and the Apify CLI's installers for claude-code, vscode and other targets. A pricing agent can call the Actor as a tool mid-conversation to check a competitor's current offer set before answering "is this a good price right now."

📊 Export to spreadsheets or CRM

Use the Actor's own shoppingResults dataset view (or Apify's CSV export) to get a flat sheet: searchQueryTerm → Keyword, shoppingResultsCount → Offers, and each offer's merchant/priceText/rating as additional columns. Note that nested arrays like organicResults and shoppingResults serialize as raw JSON text inside a CSV cell — use the dedicated dataset views, or a short unwrap script, if you need one row per offer instead of one row per keyword.

---

Is it legal to scrape Google Search results?

Yes. This Actor collects data Google already serves to any visitor of google.com and its Shopping surface — organic titles and links, People Also Ask questions, and shopping offers such as merchant name and price — without a login or any access-control bypass. These are business and product listings, not personal data, so the relevant framework is Google's Terms of Service and general database-rights considerations rather than GDPR. Review Google's Terms of Service before large-scale or commercial use, avoid re-publishing scraped content as your own, and consult legal counsel for commercial applications involving bulk storage of personal data.

---

❓ FAQ

Do the shopping prices and offers go stale?

No caching happens — every run scrapes live at request time, so priceText, rating and deliveryText reflect what Google showed at that moment. ⚠️ Because offers have no stable ID (see Shopping offer fields above), compare runs by title+merchant, not row position, when checking whether a price actually changed.

Can I get shopping offers along with the organic results?

Yes, by default. includeShoppingResults is true out of the box, and offers are fetched once per keyword (not once per page) and attached identically to every page row and the merged row via shoppingResults[], shoppingResultsCount and related fields.

How accurate is the price data?

The Actor returns each price exactly as Google's Shopping card printed it at request time — no currency conversion, and any field Google didn't print stays null instead of a fabricated 0. Accuracy after that depends on how current the merchant keeps their own Google Shopping feed; re-run before using a price in a purchase decision.

How many results can I get per run?

Organic pages are controlled by limit (10–100, or all, ~10 results per page). Shopping offers are controlled separately by maxShoppingResultsPerQuery, capped at 200 per keyword; Google serves roughly 40 offer cards per Shopping page, so the Actor walks up to six &start= pages to reach that cap.

How does the item condition filter work?

shoppingCondition (new, used, refurbished) is matched against a filter chip Google prints on the Shopping page and followed via its shoprs= token — condition is not a plain URL parameter. If Google doesn't offer that chip for a keyword, the unfiltered offers are returned and shoppingConditionApplied stays null, so you can detect when the filter didn't actually apply.

What happens when Google blocks the request?

The Actor checks for Google's /sorry/ redirect and for the absence of any expected result container (not just a keyword match, which a JS-challenge page can fake at HTTP 200). On a block it escalates from the Google SERP proxy to residential proxy sessions and retries; if a keyword still fails, it is pushed as an uncharged accounting row with errorReason set, rather than silently dropped or silently billed.

Does Google Search Results (SERP) Scraper & Shopping Prices work with Claude, ChatGPT, and AI agent frameworks?

Yes. It's reachable via Apify's MCP server (https://mcp.apify.com?tools=captivated_transformer/google-search-results-serp-scraper-and-shopping-prices) for MCP-native clients like Claude Desktop and Cursor, and callable as a plain HTTP endpoint by any agent framework through the Apify API or apify-client.

How does this compare to other Google scrapers?

As observed on their Apify Store listings on 2026-08-16: a near-identical organic-only actor, Google Search Results (SERP) Scraper, returns organic results, position, title, URL and description but documents no shopping data. A dedicated shopping actor, Google Shopping Scraper — Product Prices & Merchants, returns 13 flat offer fields (title, price, merchant, rating, etc.) but explicitly has no organic results and points users to a separate actor for those. This Actor is the one that merges both — organic SERP and priced shopping offers — into a single row per keyword.

Can I use this Actor without a Google API key or developer account?

Yes. No Google account, API key, or Custom Search Engine configuration is needed — only an Apify account to run the Actor. Proxy sessions (Google SERP, escalating to residential on a block) are managed internally by the Actor itself.

---

Conclusion

Google Search Results (SERP) Scraper & Shopping Prices turns a list of keywords into structured JSON that carries the organic SERP and the priced Google Shopping offers for that keyword on the same row — no Custom Search Engine, no separate shopping scraper, no manual join. It's built for price monitors, SEO and PPC agencies auditing client keywords across markets, and market researchers comparing shopping presence by country. Every row documents its own charge status, condition-filter outcome, and block/retry state, so nothing is billed or dropped silently. Start a run from the Apify Console with a single keyword field, or call it through the API or MCP as shown above.