Google Search Results (SERP) Scraper

Google Search Results (SERP) Scraper

The Google Search Results (SERP) Scraper is a production-ready serp scraping tool that collects structured Google Search data at scale — organic results, ads, shopping products, People Also Ask, related queries, and optional AI overview — in clean JSON. Built for marketers, developers, data analysts, and researchers, this google serp scraper solves the pain of manual SERP checks by automating reliable serp data extraction across countries, languages, and time filters. Use it to scrape Google search results for SEO, PPC, and competitive intelligence workflows — and power downstream dashboards, analytics, or automation.

What data / output can you get?

Below are the real output fields the actor pushes to the dataset (one item per query/page). You can export results from the Apify dataset in JSON/CSV.

Data field	Description	Example value
searchQuery.term	Final query string sent to Google, including filters	"nike shoes"
searchQuery.url	SERP page URL fetched	"https://www.google.com/search?q=nike+shoes&gl=US&hl=en"
searchQuery.page	Page index or "all" (for merged row)	1
searchQuery.countryCode	Country (GL) used for results	"US"
searchQuery.languageCode	UI language (HL)	"en"
resultsTotal	Parsed “About X results” count (best effort)	12500000
url	The SERP URL of the fetched page	"https://www.google.com/..."
hasNextPage	Whether Google indicates a next page	true
serpProviderCode	Provider code marker	"O"
relatedQueries[]	Top related queries from organic titles as Google links	[{"title":"Nike store","url":"https://www.google.com/search?q=Nike+store..."}]
organicResults[]	Organic results with title, url, displayedUrl, description, position, siteLinks	[{"title":"Nike Official","url":"https://www.nike.com", ...}]
paidResults[]	Paid ads with title, final url, displayedUrl, position	[{"title":"Shop Nike®","url":"https://example.com", ...}]
paidProducts[]	Shopping product cards with title, displayedUrl, prices	[{"title":"Nike Air Max","prices":["$129.99"]}]
peopleAlsoAsk[]	People Also Ask blocks (question/title/url)	[{"question":"Are Nike shoes durable?"}]
suggestedResults[]	Suggested items synthesized from related queries	[{"title":"Nike store","type":"organic","position":1}]
aiOverview	Optional AI overview text + references (if configured)	{"text":"Nike ...", "references":[{"title":"...","url":"..."}]}
customData	Flags for Perplexity/ChatGPT search and enrichment limits	{"perplexitySearch":{"enablePerplexity":false,...}}
htmlSnapshotUrl	Key for saved HTML snapshot (if enabled)	"nike_20260331_101530_p1.html"
html	Raw HTML for collected pages (if enabled)	"..."
_htmlPayloads	Array of KV store keys + HTML (if enabled)	[["nike_..._p1.html","..."]]

Notes:

• organicResults items include emphasizedKeywords, siteLinks, productInfo, and position; when enabled, icon may appear.
• peopleAlsoAsk includes fields: answer (null), question, title, url, date (null).
• The actor can push an extra merged row per keyword if include_merged is true.
• Dataset export supports JSON/CSV from the Apify UI.

Key features

• 🚀 Parallelized AI overview fetch (optional)

  • On page 1, the actor can fetch aiOverview in parallel via SerpApi (if provided), then merges it into the same dataset item.

• 🌍 Localization & language controls

  • Use country (GL), UI language (HL), language restriction (LR), and country restriction (CR) to tailor results per market.

• ⏱️ Time filters with TBS

  • Pass advanced time filters like qdr:d or custom ranges (cdr:1,cd_min:MM/DD/YYYY,cd_max:MM/DD/YYYY) to focus on recency.

• 📦 Merged “all pages” row

  • Enable include_merged to automatically add one extra dataset item per keyword combining organic results across pages.

• 🧭 Robust pagination with next-page detection

  • The scraper follows pagination and sets hasNextPage based on the pnnext selector.

• 🛡️ Smart proxies for Google SERP scraping

  • Uses the Apify GOOGLE_SERP proxy by default; on blocks, escalates to RESIDENTIAL with up to 3 retries and then sticks for the run.

• 💾 HTML snapshots (optional)

  • Save HTML to the dataset (html) and/or Key-Value Store (htmlSnapshotUrl, _htmlPayloads) for auditing and re-parsing.

• 🧪 Accurate organic/ads/products/PAA parsing

  • Extracts structured organic results, paid ads, shopping cards, and People Also Ask reliably — ideal for google search results parser use cases.

• 🧰 Developer friendly

  • Built in Python on Apify Actors. Access results via the Apify dataset API for integration into pipelines, dashboards, and scripts (e.g., serp scraper python workflows).

• 🔗 Workflow-ready

  • Perfect for rank tracking, PPC monitoring, competitive research, and content planning; pairs with your existing “google serp api” pipelines or automation tools.

How to use Google Search Results (SERP) Scraper — step by step

1. Sign in to your Apify account and open the actor in the Apify Console.
2. In the Input tab, enter your queries in keyword — one full search query or a full Google Search URL per line.
3. Optionally adjust limit (e.g., "10", "50", "all") and include_merged to control pagination and merged rows.
4. Set localization: country (GL), or override gl; pick HL (interface language), and optionally set LR/CR for content/country restrictions.
5. Add time filtering with tbs (e.g., qdr:w or a custom cdr range) to scope by recency.
6. If needed, choose proxy_location ("us" or "ca") to control proxy region during google serp crawling.
7. Click Start. Monitor the Log for progress, page counts, and any proxy escalation to residential.
8. Go to the Output tab to view results and export the dataset in JSON/CSV.

Pro Tip: Turn on include_merged to get a compact “all pages” roll-up per keyword for downstream analysis or BI tooling.

Use cases

Use case	Description
SEO rank tracking	Automate google search results scraper runs across markets to monitor rankings and SERP layout changes.
PPC & PLA monitoring	Track paidResults and paidProducts for competitive ad coverage and pricing intelligence.
Content research	Mine peopleAlsoAsk and relatedQueries to inform content briefs and Answer Engine Optimization.
Competitor analysis	Compare organicResults across regions/languages to benchmark visibility and messaging.
Local SEO	Localize with GL/HL/LR/CR to evaluate SERPs for specific locales and languages.
Data pipelines	Feed structured SERP JSON into analytics stacks via the Apify dataset API.
Academic/market research	Collect reproducible serp data extraction for studies on query intent and result diversity.

Why choose Google Search Results (SERP) Scraper?

• 🎯 Precision SERP data: Structured organic, ads, shopping, and PAA that’s ready to query and analyze.
• 🌐 True localization: Control GL, HL, LR, CR, and time filters (TBS) for market-specific SERPs.
• ⚙️ Built for scale: Batch queries, paginate reliably, and optionally produce a merged “all pages” record per keyword.
• 👨‍💻 Developer-first: Access clean JSON via the Apify dataset for API pipelines and serp scraper python workflows.
• 🛡️ Reliable proxying: Default GOOGLE_SERP proxy with automatic escalation to residential on block — production-grade resiliency.
• 💾 Auditable: Optional HTML snapshots saved to dataset and/or Key-Value Store for verification and re-parsing.
• 💡 Smarter insights: Optional aiOverview field (when configured) adds context for research and summaries.

Unlike brittle browser extensions, this google serp crawler is engineered for consistency, localization control, and automation-readiness.

Is it legal / ethical to use this scraper?

Yes — when done responsibly. The actor collects data from publicly available Google Search result pages. You are responsible for:

• Respecting Google’s Terms of Service and robots.txt.
• Complying with applicable laws and regulations (e.g., privacy and data protection).
• Using reasonable rates and appropriate proxies for google serp scraping to avoid abuse. This tool does not access private or authenticated data.

Input parameters & output format

Example JSON input

{
  "keyword": "nike shoes\nbest laptop 2025",
  "include_merged": true,
  "limit": "all",
  "start": 0,
  "page": 1,
  "country": "US",
  "gl": "US",
  "hl": "en",
  "tbs": "qdr:w",
  "lr": "lang_en",
  "cr": "countryUS",
  "proxy_location": "us"
}

Input parameter reference

• keyword (string, required) — Enter one full search query per line. Default: "nike".
• include_merged (boolean, optional) — If enabled, adds one extra row per keyword that merges results from all pages ("page" = "all"). Default: true.
• limit (string, optional) — Each page returns up to 10 results. Use "all" to fetch as many pages as possible (up to a safe limit). Enum: "10","20","30","40","50","100","all". Default: "all".
• start (integer, optional) — Optional offset for the first result index (Google start parameter).
• page (integer, optional) — If set, fetches only this page number (1 = first page, 2 = second, …).
• country (string, optional) — Main country selector. Automatically sets Google gl and default proxy location. Uses ISO 3166 A-2 codes.
• gl (string, optional) — Override the local country for results (Google gl parameter). ISO 3166 A-2.
• hl (string, optional) — Google UI language (e.g., en, fr, de).
• tbs (string, optional) — Advanced time filter (e.g., qdr:d, qdr:w, or cdr:1,cd_min:MM/DD/YYYY,cd_max:MM/DD/YYYY).
• lr (string, optional) — Language restriction by content (Google lr), e.g., lang_en.
• cr (string, optional) — Country restriction (Google cr), e.g., countryUS.
• proxy_location (string, optional) — Force the proxy region: "us" or "ca".

Example JSON output (one dataset item)

{
  "searchQuery": {
    "term": "nike shoes",
    "url": "https://www.google.com/search?q=nike+shoes&gl=US&hl=en",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "nike shoes",
  "url": "https://www.google.com/search?q=nike+shoes&gl=US&hl=en",
  "hasNextPage": true,
  "serpProviderCode": "O",
  "resultsTotal": 12500000,
  "relatedQueries": [
    {
      "title": "Nike store",
      "url": "https://www.google.com/search?q=Nike+store&hl=en&gl=US"
    }
  ],
  "paidResults": [
    {
      "title": "Shop Nike®",
      "url": "https://example.com/landing",
      "displayedUrl": "example.com/landing",
      "description": "",
      "emphasizedKeywords": ["nike shoes"],
      "siteLinks": [],
      "productInfo": {},
      "type": "paid",
      "position": 1
    }
  ],
  "paidProducts": [
    {
      "title": "Nike Air Max",
      "displayedUrl": "store.example.com/product",
      "prices": ["$129.99"]
    }
  ],
  "aiOverview": null,
  "organicResults": [
    {
      "title": "Nike. Just Do It. Nike.com",
      "url": "https://www.nike.com",
      "displayedUrl": "www.nike.com",
      "description": "Shop the latest Nike shoes, apparel and more.",
      "emphasizedKeywords": ["nike shoes"],
      "siteLinks": [],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    {
      "title": "Nike store",
      "url": "https://www.google.com/search?q=Nike+store&hl=en&gl=US",
      "type": "organic",
      "position": 1
    }
  ],
  "peopleAlsoAsk": [
    {
      "answer": null,
      "question": "Are Nike shoes durable?",
      "title": "Are Nike shoes durable?",
      "url": "https://example.com/faq",
      "date": null
    }
  ],
  "customData": {
    "perplexitySearch": {
      "enablePerplexity": false,
      "searchRecency": null,
      "returnImages": false,
      "returnRelatedQuestions": false
    },
    "chatGptSearch": {
      "enableChatGpt": false
    },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": null
}

Notes:

• html and _htmlPayloads appear only if HTML saving is enabled.
• aiOverview is included when the AI overview fetch is configured and successful.

FAQ

Does it use proxies automatically?

Yes. By default, it uses the Apify GOOGLE_SERP proxy. If Google blocks the request, the actor escalates to a RESIDENTIAL proxy with up to 3 retries and then continues with residential for the rest of the run. You can also force proxy_location ("us" or "ca").

Can it scrape Google People Also Ask?

Yes. The actor parses People Also Ask blocks and outputs them under peopleAlsoAsk with question/title/url fields. This is useful for content research and “scrape google people also ask” workflows.

Does it capture paid ads and shopping products?

Yes. Paid ads are returned under paidResults and shopping cards under paidProducts with titles, links (or displayedUrl), and prices when available.

How many results per page can I get?

Google typically returns ~10 organic results per page. Use limit to control how many pages to fetch, or set include_merged to add one merged “all pages” item per keyword.

Can I localize results by country and language?

Yes. Use country/gl for local country results, hl for the Google UI language, and optionally lr (language restriction) and cr (country restriction) to refine SERPs.

Does it provide an AI overview?

It can. When configured, the actor fetches an aiOverview (text and references) alongside the first page, useful for summary-driven analysis in serp data extraction pipelines.

What export formats are supported?

You can view and export the dataset from the Apify Console in JSON/CSV. This makes it easy to integrate with analytics tools or a google results scraper python workflow.

Is there a free trial or pricing?

The actor runs under your Apify account and plan. Check the Apify Store listing for current pricing and trial options, then run it directly from the console or via the Apify API.

Closing thoughts

The Google Search Results (SERP) Scraper is built for reliable, scalable SERP data collection. It delivers structured organic, ads, shopping, PAA, related queries, and optional AI overview — all in one streamlined google search results scraper. Marketers, developers, and analysts can automate market tracking, PPC audits, and content research with clean JSON/CSV outputs. For engineering teams, connect the Apify dataset to your pipelines to scrape google search results without api lock-in and power a robust google serp crawler workflow. Start extracting smarter SERP insights at scale today.