Google Search Results (SERP) Scraper

Google Search Results (SERP) Scraper

The Google Search Results (SERP) Scraper is a production-ready Google SERP scraper that collects structured organic results, ads, shopping products, People Also Ask, related and suggested queries, and optional AI overview — all by keyword and location. It helps you scrape Google search results reliably with proxy handling and return a clean, standardized SERP JSON for SEO monitoring, rank tracking, and analysis. Built for marketers, developers, data analysts, and researchers, it enables automated Google SERP scraping at scale with localization and time filtering.

What data / output can you get?

Below are the main fields this Google search results scraper returns in each dataset item, with examples:

Data type	Description	Example value
searchQuery.term	Final query string sent to Google (with applied operators/filters)	"nike shoes"
searchQuery.countryCode	Country code used for SERP localization (GL)	"US"
url	Final SERP URL for the query	"https://www.google.com/search?q=nike+shoes&gl=us&hl=en"
resultsTotal	Parsed “About X results” count (if available)	123000000
organicResults[].title	Organic result title	"Nike. Just Do It. Nike.com"
organicResults[].url	Organic result URL (clean, external)	"https://www.nike.com/"
organicResults[].displayedUrl	Display-friendly URL	"www.nike.com"
organicResults[].position	Organic rank position (de-duplicated across pages)	1
paidResults[].title	Paid ad title (if any)	"Shop Nike® Official Store"
paidResults[].url	Paid ad destination URL (cleaned)	"https://store.nike.com/..."
paidProducts[].prices	Detected price strings from shopping blocks	["$129.99", "$119.00"]
peopleAlsoAsk[].question	People Also Ask question (if present)	"Are Nike shoes worth it?"
relatedQueries[].title	Related/suggested query title (built from top organic titles)	"Nike running shoes"
aiOverview.text	Optional AI overview text (via external API if enabled)	"Nike offers a range of footwear..."
suggestedResults[]	Suggested results derived from relatedQueries	[{"title":"Nike running shoes","url":"https://www.google.com/search?..."}]
customData	Extra configuration flags (Perplexity/ChatGPT/leads enrichment)	{"perplexitySearch": {...}}
htmlSnapshotUrl	Key of saved HTML snapshot in KV store (when enabled)	"nike_shoes_20260413_101530_p1.html"
hasNextPage	Whether the scraper detected a next page link for the query	false
serpProviderCode	Internal SERP provider code	"O"

Notes:

• Results are written to the Apify dataset and can be exported to JSON/CSV.
• When enabled and configured, aiOverview contains summarized text and source references. If not enabled, aiOverview will be null.

Key features

• 🔒 Anti-blocking proxy strategy (SERP-ready) Uses Apify’s GOOGLE_SERP proxy by default and automatically switches to RESIDENTIAL with retries if Google blocks the request. It then sticks to residential for the remainder of the run for stability.

• 📦 Batch scraping & bulk automation Paste multiple queries (one per line) to scrape Google search results at scale. This Google SERP scraper tool pushes results page-by-page in real time.

• 🌍 Localized Google SERP data Control country (GL), UI language (HL), language restriction (LR), and country restriction (CR). Supports proxy location selection to control where requests originate.

• ⏱️ Time filters (TBS) Apply advanced time filtering with tbs (e.g., qdr:d for past day, or custom ranges via cdr:1, cd_min, cd_max) to target fresh or historical SERPs.

• 🧠 Optional AI overview When enabled with an external key, captures Google’s AI overview text and references into aiOverview for Answer Engine Optimization tracking.

• 🛍️ Ads, Shopping, and People Also Ask Extract paidResults, paidProducts (detected prices), and peopleAlsoAsk questions alongside organicResults, relatedQueries, and suggestedResults.

• 🧩 Merged “all pages” row Enable include_merged to get one extra dataset item per keyword that merges organic results across pages (page = "all").

• 🧾 HTML snapshots Save HTML to the dataset or to the key-value store and get htmlSnapshotUrl for auditing and downstream parsing.

• 👨‍💻 Developer-friendly SERP JSON Clean, consistent keys: searchQuery, organicResults, paidResults, paidProducts, peopleAlsoAsk, relatedQueries, suggestedResults, aiOverview, customData, and more. Perfect for pipelines in Python/Node.js via the Apify API.

• 🌐 Google SERP scraping with proxies Built for automated Google SERP scraping workloads, including local targeting and reliable proxy handling.

How to use Google Search Results (SERP) Scraper - step by step

1. Sign in to Apify Go to console.apify.com and log in (or create a free account).

2. Open the actor Search for “Google Search Results (SERP) Scraper” in the Apify Store or your Actors list and open it.

3. Add your keywords In the Input tab, paste your search queries into the “Search Keyword(s)” textarea — one full query per line. You can also paste full Google Search URLs.

4. Set pagination/limits Choose “Number of results” (limit). “all” fetches as many pages as possible up to a safe maximum. Optionally set Start (Google start offset) or Page (fetch only a specific page).

5. Localize the SERP Pick Country (GL), UI language (HL), and optionally set LR (language restriction) and CR (country restriction). You can also force a proxy_location (us/ca).

6. Control time filtering (optional) Use TBS to apply time filters (e.g., qdr:w for past week, or a custom CDR range).

7. Start the run Click Start. Watch the Log tab to see progress and automatic proxy fallback (to RESIDENTIAL) if any block is detected.

8. Download or integrate results Open the Dataset tab to browse results. Export to JSON or CSV, or connect via the Apify API for programmatic access.

Pro Tip: Enable merged output (include_merged) to add a single, combined row per keyword with organicResults across all pages for consolidated analysis.

Use cases

Use case name	Description
SEO rank tracking	Collect organicResults with positions and URLs to monitor keyword rankings over time with localized Google SERP data.
Competitor research	Extract titles, descriptions, and displayed URLs for top competitors to analyze SERP strategy and messaging.
Ads & shopping monitoring	Capture paidResults and paidProducts with price detection to track advertising visibility and retail pricing trends.
Answer Engine Optimization	Store peopleAlsoAsk and aiOverview (when enabled) to optimize content for zero-click/AI answers in SERPs.
Content ideation	Use relatedQueries and suggestedResults to discover adjacent topics and expand keyword clusters.
Regional SERP analysis	Compare results across countries and languages with GL/HL/LR/CR to understand localization impact.
Data pipelines via API	Stream SERP JSON into analytics or ETL workflows through the Apify API for automated reporting.
Academic & market research	Collect SERP snapshots over time with HTML snapshots and resultsTotal for longitudinal studies.

Why choose Google Search Results (SERP) Scraper?

This Google search results scraper focuses on precision, reliability, and scale for SERP data extraction.

• 🎯 Accurate SERP structure: Outputs standardized fields (searchQuery, organicResults, paidResults, peopleAlsoAsk, relatedQueries, suggestedResults, aiOverview, customData).
• 🌍 Localization controls: GL/HL/LR/CR parameters and proxy_location for consistent geographic targeting.
• ⚡ Built for scale: Batch queries, page-by-page streaming to the dataset, and an optional merged “all pages” row per keyword.
• 🔌 Developer access: Integrate easily via the Apify API; the JSON schema is consistent and automation-friendly (Google search results scraping API workflows).
• 🛡️ Robust proxy strategy: Uses GOOGLE_SERP proxy and auto-switches to RESIDENTIAL on blocks — more stable than ad-hoc browser extensions or manual scraping.
• 💾 Auditable runs: Optional HTML snapshots (dataset or key-value store) to validate and reproduce results.
• 💸 Cost-effective automation: Designed for automated Google SERP scraping and repeatable data collection.

In short, it’s a Google SERP scraper tool purpose-built for automated, localized, and structured Google SERP data extraction — without the instability of one-off scripts.

Is it legal / ethical to use Google Search Results (SERP) Scraper?

Yes — when done responsibly. This actor collects data from publicly available Google Search result pages.

Guidelines:

• Only use data from public SERPs. Do not attempt to access private or authenticated content.
• Comply with Google’s Terms of Service and robots.txt rules.
• Ensure your use adheres to applicable laws and regulations (e.g., GDPR, CCPA).
• Use reasonable limits and respect anti-blocking measures (e.g., proxy fallback) to avoid abusive behavior.
• Consult your legal team for edge cases or jurisdiction-specific requirements.

Input parameters & output format

Example JSON input

{
  "keyword": "nike shoes\nbest running shoes 2026",
  "include_merged": true,
  "limit": "20",
  "start": 0,
  "page": 1,
  "country": "US",
  "gl": "US",
  "hl": "en",
  "tbs": "qdr:w",
  "lr": "lang_en",
  "cr": "countryUS",
  "proxy_location": "us"
}

Input fields

• keyword (string, required)

  • Description: Enter one full search query per line. Each line is a separate search (e.g., “nike shoes”, “best laptop 2025”).
  • Default: "nike"
  • Required: Yes

• include_merged (boolean)

  • Description: If enabled, the dataset will include one extra row per keyword that merges results from all pages (page = "all").
  • Default: true
  • Required: No

• limit (string)

  • Description: Each page returns up to 10 results. Use "all" to fetch as many pages as possible (up to a safe limit).
  • Default: "all"
  • Required: No

• start (integer)

  • Description: Optional offset for the first result index (Google start parameter). Leave empty for the default.
  • Default: —
  • Required: No

• page (integer)

  • Description: If set, fetches only this page number (1 = first page, 2 = second page, …).
  • Default: —
  • Required: No

• country (string)

  • Description: Main country selector. Automatically sets Google gl and default proxy location. Uses ISO 3166 A‑2 country codes.
  • Default: —
  • Required: No

• gl (string)

  • Description: Override the local country for results (Google gl parameter). Uses ISO 3166 A‑2 codes.
  • Default: —
  • Required: No

• hl (string)

  • Description: Google Search interface language (e.g., en, fr, de).
  • Default: —
  • Required: No

• tbs (string)

  • Description: Advanced time filter. Examples: qdr:d (past day), qdr:w (week), or custom range: cdr:1,cd_min:01/01/2024,cd_max:01/31/2024.
  • Default: —
  • Required: No

• lr (string)

  • Description: Restrict results by content language (Google lr parameter), e.g., lang_en, lang_fr.
  • Default: —
  • Required: No

• cr (string)

  • Description: Restrict results by country of origin (Google cr parameter), e.g., countryUS, countryFR.
  • Default: —
  • Required: No

• proxy_location (string)

  • Description: Force the proxy region. Useful if you want to keep Google country separate from the proxy location. Options: us, ca.
  • Default: —
  • Required: No

Example JSON output

{
  "searchQuery": {
    "term": "nike shoes",
    "url": "https://www.google.com/search?q=nike+shoes&gl=us&hl=en",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "nike shoes",
  "url": "https://www.google.com/search?q=nike+shoes&gl=us&hl=en",
  "hasNextPage": false,
  "serpProviderCode": "O",
  "resultsTotal": 123000000,
  "relatedQueries": [
    {
      "title": "Nike running shoes",
      "url": "https://www.google.com/search?q=Nike+running+shoes&hl=en&gl=us"
    }
  ],
  "paidResults": [
    {
      "title": "Shop Nike® Official Store",
      "url": "https://store.nike.com/",
      "displayedUrl": "store.nike.com",
      "description": "",
      "emphasizedKeywords": ["nike shoes"],
      "siteLinks": [],
      "productInfo": {},
      "type": "paid",
      "position": 1
    }
  ],
  "paidProducts": [
    {
      "title": "Nike Air Zoom Pegasus",
      "displayedUrl": "www.example-retailer.com/product/pegasus",
      "prices": ["$119.99"]
    }
  ],
  "aiOverview": {
    "text": "Nike offers a wide range of athletic footwear...",
    "references": [
      {
        "title": "Nike Official",
        "url": "https://www.nike.com/"
      }
    ]
  },
  "organicResults": [
    {
      "title": "Nike. Just Do It. Nike.com",
      "url": "https://www.nike.com/",
      "displayedUrl": "www.nike.com",
      "description": "Discover the latest Nike shoes and gear.",
      "emphasizedKeywords": ["nike shoes"],
      "siteLinks": [],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    {
      "title": "Nike running shoes",
      "url": "https://www.google.com/search?q=Nike+running+shoes&hl=en&gl=us",
      "type": "organic",
      "position": 1
    }
  ],
  "peopleAlsoAsk": [
    {
      "answer": null,
      "question": "Are Nike shoes worth it?",
      "title": "Are Nike shoes worth it?",
      "url": "https://www.example.com/review",
      "date": null
    }
  ],
  "customData": {
    "perplexitySearch": {
      "enablePerplexity": false,
      "searchRecency": null,
      "returnImages": false,
      "returnRelatedQuestions": false
    },
    "chatGptSearch": {
      "enableChatGpt": false
    },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": null
}

Fields that may be null:

• aiOverview when AI mode isn’t enabled
• htmlSnapshotUrl when HTML saving is disabled
• locationUule when not provided

FAQ

Does this Google SERP scraper support bulk queries?

Yes. Paste multiple queries (one per line) in the “Search Keyword(s)” field. Each line is processed as an individual Google search, with results streamed to the dataset as pages are scraped.

How are proxies handled to avoid blocks?

By default, the actor uses the GOOGLE_SERP proxy. If Google responds with a block (e.g., “unusual traffic”), it automatically switches to RESIDENTIAL proxies and retries. Once on residential, it sticks to it for the rest of the run.

Can I localize results by country and language?

Yes. Use Country/GL for localized results, HL for Google’s interface language, and optionally LR for content language and CR for country restriction. You can also force a proxy_location (us/ca) for traffic origination.

How many results per query can I extract?

Use the “Number of results” (limit) selector. Each Google page typically contains up to ~10 organic results. “all” attempts to fetch as many pages as possible up to a safe maximum.

Can I filter results by time?

Yes. Use the TBS (Time filter) parameter (e.g., qdr:d for past day, qdr:w for past week, or a custom CDR range). The actor maps TBS into the proper Google query parameters.

What data types are included beyond organic results?

In addition to organicResults, the actor collects paidResults (ads), paidProducts (shopping blocks with price detection), peopleAlsoAsk, relatedQueries, suggestedResults, and optionally aiOverview when enabled.

How do I export the results?

Open the run’s Dataset and export in JSON or CSV. You can also fetch results programmatically using the Apify API for integration into analytics or pipelines.

Is it legal to scrape Google search results?

Yes, when done responsibly. The actor accesses public SERP pages only. Users must comply with Google’s Terms of Service and applicable laws (e.g., GDPR/CCPA) and avoid abusive behavior.

Closing CTA / Final thoughts

The Google Search Results (SERP) Scraper is built for fast, reliable Google SERP data extraction at scale. It returns structured JSON covering organic results, ads, shopping products, People Also Ask, and related/suggested queries — with localization, time filtering, and resilient proxy handling. Ideal for marketers, developers, and analysts, it plugs into dashboards and pipelines via the Apify dataset and API. Start scraping Google search results efficiently and power your SEO monitoring, competitive analysis, and research with clean, standardized SERP data.