Google Search Results Scraper

Google Search Results Scraper

The Google Search Results Scraper is a production-ready Google SERP scraper that captures organic results, paid ads, shopping products, People Also Ask, related and suggested queries, and optional AI Overviews in structured JSON. Designed for marketers, developers, data analysts, and researchers, this Google search results extractor solves the challenge of scaling SERP research with location, language, and date filters — all exportable to JSON/CSV for analytics. Use this Google SERP data scraper to power SEO monitoring, PPC analysis, competitive research, and content strategy at scale.

What data / output can you get?

Below are the primary fields produced per query in the dataset. Each item follows a standard SERP JSON structure and can be exported via Apify in JSON/CSV.

Data type	Description	Example value
searchQuery.term	Final query string sent to Google (with applied filters/operators)	"javascript intitle:"tutorial""
searchQuery.countryCode	Country code used for the query (uppercased)	"US"
searchQuery.languageCode	Interface language (hl) used for the query	"en"
url	Final SERP URL for the query	"https://www.google.com/search?q=javascript&hl=en&gl=us"
resultsTotal	Parsed “About X results” number (if available)	12500000
organicResults[]	Array of organic listings with metadata	[{"title":"…","url":"…","displayedUrl":"…","description":"…","emphasizedKeywords":["javascript"],"siteLinks":[{"title":"…","url":"…","description":""}], "productInfo":{},"type":"organic","position":1}]
paidResults[]	Array of paid ads (if any)	[{"title":"…","url":"…","displayedUrl":"…","description":"","emphasizedKeywords":["javascript"],"siteLinks":[],"productInfo":{},"type":"paid","position":1}]
paidProducts[]	Shopping/product grid cards (if any)	[{"title":"Best JS course","displayedUrl":"example.com","prices":["$29.99"]}]
peopleAlsoAsk[]	PAA questions with optional link	[{"question":"What is JavaScript?","title":"What is JavaScript?","url":"https://…","answer":null,"date":null}]
relatedQueries[]	Related query suggestions with Google links	[{"title":"javascript tutorial","url":"https://www.google.com/search?q=javascript+tutorial&hl=en&gl=us"}]
suggestedResults[]	Suggestion list derived from relatedQueries (typed + position)	[{"title":"javascript tutorial","url":"https://…","type":"organic","position":1}]
aiOverview	AI Overview text and references (if enabled)	{"text":"…summary…","references":[{"title":"MDN Web Docs","url":"https://developer.mozilla.org/…"}]}
customData	Echo of add-on toggles and enrichment limits	{"perplexitySearch":{"enablePerplexity":false,"searchRecency":null,"returnImages":false,"returnRelatedQuestions":false}, "chatGptSearch":{"enableChatGpt":false}, "maximumLeadsEnrichmentRecords":0,"leadsEnrichmentDepartments":[]}
hasNextPage	Indicates if Google shows a next page	true

Notes:

• Bonus fields: html (raw HTML, if saveHtml=true) and htmlSnapshotUrl (link to HTML saved in key-value store, if saveHtmlToKeyValueStore=true).
• Export formats: Use the Apify UI to download JSON/CSV from the dataset.

Key features

• ⚙️ Robust proxy strategy with fallback
  Always uses Apify proxies with the GOOGLE_SERP group by default and automatically falls back to RESIDENTIAL proxies (with retries) on blocks — then sticks to residential for remaining requests for higher success rates.

• 📦 Bulk input & real-time streaming
  Paste multiple search terms or full Google search URLs (one per line). The scraper pushes structured SERP data to the dataset after each page for near real-time visibility.

• 🧠 Optional AI Overviews via SerpApi
  Enable aiMode and provide serpApiKey to enrich each query with Google AI Overview (text + references) for AEO and GEO tracking.

• 📈 Paid ads & product listings
  Capture paidResults (Google Ads) and paidProducts (shopping grids). Enable focusOnPaidAds for additional retries via an ad-specialized proxy when ads are likely present.

• 🎯 Advanced Google filters
  Apply site/related filters, exact match, words in title/text/URL, file types, quick date ranges, and absolute/relative before/after dates to refine SERPs precisely.

• 🌍 Geo & language targeting
  Control country (gl), interface language (hl), searchLanguage (lr), and even exact location via locationUule for accurate local SERPs. Toggle mobileResults for mobile SERPs.

• 🧪 HTML snapshots for QA
  Save HTML to the dataset (html) and/or save HTML to the key-value store (htmlSnapshotUrl) for debugging, auditing, or custom parsing.

• 📊 Analytics-ready outputs
  Structured, consistent SERP JSON with organicResults, paidResults, peopleAlsoAsk, relatedQueries, and suggestedResults — easy to export to JSON/CSV for dashboards and data pipelines.

How to use Google Search Results Scraper - step by step

1. Sign in to Apify and open the Google Search Results Scraper actor.
2. In the Input tab, paste queries into the queries field — one search term or full Google Search URL per line.
3. Set resultsPerPage and maxPagesPerQuery to control volume (roughly ~10 organic results per page).
4. Configure location and language: countryCode, languageCode, optional searchLanguage and locationUule.
5. (Optional) Add filters: forceExactMatch, site, relatedToSite, wordsInTitle/wordsInText/wordsInUrl, quickDateRange/beforeDate/afterDate, fileTypes.
6. (Optional) Enable add-ons: aiMode + serpApiKey for AI Overviews, focusOnPaidAds for ad detection, saveHtml/saveHtmlToKeyValueStore for HTML snapshots.
7. (Optional) Proxies: Leave the default proxyConfiguration (GOOGLE_SERP) or supply your own Apify proxy groups.
8. Click Start. Watch the Log for progress and any proxy fallback messages. Results will stream to the default dataset.
9. Export your dataset as JSON/CSV from the Output tab.

Pro Tip: For debugging SERP layouts or parsers, enable saveHtmlToKeyValueStore to get a browser-viewable htmlSnapshotUrl per page.

Use cases

Use case name	Description
SEO teams – SERP monitoring	Track organic rankings, featured blocks, and People Also Ask for target keywords to inform optimization roadmaps.
PPC managers – ads intelligence	Catalog paidResults and paidProducts by query to analyze ad presence, creative variations, and competitor coverage.
Content teams – topic research	Collect organicResults and relatedQueries to identify content gaps and build briefs aligned with search intent.
Competitor analysts – market mapping	Aggregate SERP footprints by country/language to benchmark competitors across geographies and devices.
Data engineers – API pipelines	Feed standardized SERP JSON (organicResults, peopleAlsoAsk, aiOverview) into analytics or ML pipelines.
Researchers – trend tracking	Use date filters to evaluate how narratives evolve (e.g., last week vs. last year) across SERPs.
Local SEO – geo-targeted audits	Apply locationUule and countryCode to run city-level audits and compare mobile vs. desktop SERPs.

Why choose Google Search Results Scraper?

Built for precision and scale, this Google search results scraping tool combines robust proxying, rich filters, and structured outputs.

• 🎯 Accurate, structured SERP JSON: Organic, paid, products, PAA, related and suggested queries are neatly packaged for analysis.
• 🌍 Flexible geo & language control: countryCode, languageCode, searchLanguage, and exact location (UULE) for localized SERPs.
• ⚡ Scalable & reliable: Uses GOOGLE_SERP proxies by default and falls back to RESIDENTIAL on blocks (with retries).
• 🧠 AI-ready: Optional AI Overviews via aiMode + serpApiKey for Answer Engine Optimization and brand visibility analysis.
• 🔬 Transparent QA: Save HTML to dataset and/or key-value store for repeatable audits and parser evolution.
• 💻 Developer-friendly: Python-based Apify actor with standardized JSON/CSV dataset outputs for downstream automation.
• 🛡️ Safer than browser plugins: Avoid brittle extensions; rely on infrastructure designed for consistency and repeatability.

In short: a production-ready Google SERP scraper that outperforms unstable alternatives by combining robust proxying, granular filters, and analytics-ready outputs.

Is it legal / ethical to use Google Search Results Scraper?

Yes — when done responsibly. The actor collects data from publicly available Google Search result pages. You are responsible for ensuring your use complies with Google’s Terms of Service and with applicable regulations (e.g., GDPR, CCPA). Avoid scraping personal data without a legitimate basis, respect robots.txt where applicable, and consult your legal team for edge cases. Use reasonable limits and respect proxy fallback behavior to minimize friction.

Input parameters & output format

Example JSON input

{
  "queries": "javascript\npython site:docs.python.org",
  "resultsPerPage": 50,
  "maxPagesPerQuery": 2,
  "aiMode": "aiModeWithSearchResults",
  "serpApiKey": "YOUR_SERPAPI_KEY",
  "perplexitySearch": {
    "enablePerplexity": false,
    "searchRecency": "",
    "returnImages": false,
    "returnRelatedQuestions": false
  },
  "chatGptSearch": {
    "enableChatGpt": false
  },
  "maximumLeadsEnrichmentRecords": 0,
  "leadsEnrichmentDepartments": [],
  "focusOnPaidAds": false,
  "countryCode": "us",
  "searchLanguage": "en",
  "languageCode": "en",
  "locationUule": null,
  "forceExactMatch": false,
  "site": null,
  "relatedToSite": null,
  "wordsInTitle": ["tutorial"],
  "wordsInText": ["guide"],
  "wordsInUrl": ["docs"],
  "quickDateRange": "m3",
  "beforeDate": null,
  "afterDate": null,
  "fileTypes": ["pdf", "txt"],
  "mobileResults": false,
  "includeUnfilteredResults": false,
  "saveHtml": false,
  "saveHtmlToKeyValueStore": true,
  "includeIcons": false,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["GOOGLE_SERP"]
  }
}

All input fields

• queries (string, required)
  Description: One search term or full Google Search URL per line (supports Google operators).
  Default: none (required)

• resultsPerPage (integer)
  Description: Desired number of results per page (passed as num); Google may return fewer on page 1.
  Default: 100 (min 1, max 100)

• maxPagesPerQuery (integer)
  Description: Maximum pages to scrape per query (~10 organic results per page).
  Default: 1 (min 1)

• aiMode (string)
  Description: Enable Google AI Mode scraping for AEO/GEO analysis.
  Enum: aiModeOff, aiModeWithSearchResults, aiModeOnly
  Default: aiModeOff

• serpApiKey (string, nullable, secret)
  Description: Optional SerpApi key; when set and aiMode ≠ aiModeOff, fetches AI Overview via SerpApi.
  Default: null

• perplexitySearch (object)
  Description: Add-on to fetch AI answers via Perplexity Sonar.
  Properties and defaults:
  • enablePerplexity (boolean, default: false)
  • searchRecency (string, enum: "", "day", "week", "month", "year"; default: "")
  • returnImages (boolean, default: false)
  • returnRelatedQuestions (boolean, default: false)

• chatGptSearch (object)
  Description: Add-on to fetch AI answers via OpenAI’s search model (includes query fan-out in output under customData).
  Properties and defaults:
  • enableChatGpt (boolean, default: false)

• maximumLeadsEnrichmentRecords (integer)
  Description: Add-on to enrich domains with business leads; maximum leads per domain (0 disables).
  Default: 0 (min 0)

• leadsEnrichmentDepartments (array of strings)
  Description: Filter leads by department (works only if maximumLeadsEnrichmentRecords > 0).
  Default: []
  Enum: "c-suite", "product", "engineering-technical", "design", "education", "finance", "human-resources", "information-technology", "legal", "marketing", "medical-health", "operations", "sales", "consulting"

• focusOnPaidAds (boolean)
  Description: Enable paid results (ads) extraction with ad-specialized proxy retries (extra cost per page).
  Default: false

• countryCode (string)
  Description: Country selector (affects gl and Google domain).
  Default: "us" (select from the provided country list)

• searchLanguage (string, nullable)
  Description: Restrict results to a content language (lr).
  Default: null (enum includes "", "ar", "bg", …, "zh-TW")

• languageCode (string)
  Description: Interface language (hl).
  Default: "en" (enum includes "af", "sq", …, "zu")

• locationUule (string, nullable)
  Description: Exact location code (uule) for local SERPs.
  Default: null

• forceExactMatch (boolean)
  Description: Wraps query in quotes for exact phrase matching (may reduce results).
  Default: false

• site (string, nullable)
  Description: Limit results to a specific site (site:domain).
  Default: null

• relatedToSite (string, nullable)
  Description: Filter for pages related to a site (related:domain).
  Default: null

• wordsInTitle (array of strings)
  Description: Ensure words appear in title (intitle:).
  Default: [] (maxItems: 32)

• wordsInText (array of strings)
  Description: Ensure words appear in body text (intext:).
  Default: [] (maxItems: 32)

• wordsInUrl (array of strings)
  Description: Ensure words appear in the URL (inurl:).
  Default: [] (maxItems: 32)

• quickDateRange (string, nullable)
  Description: Quick date filter (qdr), e.g., d10, w1, m3, y1.
  Default: null

• beforeDate (string, nullable)
  Description: Filter results before a date; supports absolute (YYYY-MM-DD) or relative (e.g., "8 days", "3 months").
  Default: null

• afterDate (string, nullable)
  Description: Filter results after a date; supports absolute or relative dates.
  Default: null

• fileTypes (array of strings)
  Description: Restrict to specific file types (filetype:…).
  Default: [] (supports types like "pdf", "csv", "html", "docx", "ipynb", "sql", etc.)

• mobileResults (boolean)
  Description: Return mobile SERPs when true; else desktop.
  Default: false

• includeUnfilteredResults (boolean)
  Description: Include lower-quality results Google normally filters (tbs=li:1).
  Default: false

• saveHtml (boolean)
  Description: Save raw HTML to dataset under the html field.
  Default: false

• saveHtmlToKeyValueStore (boolean)
  Description: Save HTML to key-value store and write link to htmlSnapshotUrl.
  Default: true

• includeIcons (boolean)
  Description: Include icon image data in results if found.
  Default: false

• proxyConfiguration (object)
  Description: Proxy settings. By default, uses GOOGLE_SERP and falls back to RESIDENTIAL on blocks (with retries).
  Default prefill: {"useApifyProxy": true, "apifyProxyGroups": ["GOOGLE_SERP"]}

Example JSON output (single dataset item)

{
  "searchQuery": {
    "term": "javascript intitle:\"tutorial\"",
    "url": "https://www.google.com/search?q=javascript+intitle%3A%22tutorial%22&hl=en&gl=us",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "javascript intitle:\"tutorial\"",
  "url": "https://www.google.com/search?q=javascript+intitle%3A%22tutorial%22&hl=en&gl=us",
  "hasNextPage": true,
  "serpProviderCode": "O",
  "resultsTotal": 12500000,
  "relatedQueries": [
    {
      "title": "javascript tutorial",
      "url": "https://www.google.com/search?q=javascript+tutorial&hl=en&gl=us"
    }
  ],
  "paidResults": [
    {
      "title": "Learn JavaScript Fast",
      "url": "https://ads.example.com/js-course",
      "displayedUrl": "ads.example.com/js-course",
      "description": "",
      "emphasizedKeywords": ["javascript"],
      "siteLinks": [],
      "productInfo": {},
      "type": "paid",
      "position": 1
    }
  ],
  "paidProducts": [
    {
      "title": "JavaScript Course Bundle",
      "displayedUrl": "shop.example.com",
      "prices": ["$29.99"]
    }
  ],
  "aiOverview": {
    "text": "JavaScript is a versatile programming language used for web development...",
    "references": [
      { "title": "MDN Web Docs", "url": "https://developer.mozilla.org/..." }
    ]
  },
  "organicResults": [
    {
      "title": "JavaScript Tutorial - W3Schools",
      "url": "https://www.w3schools.com/js/",
      "displayedUrl": "www.w3schools.com/js/",
      "description": "Learn JavaScript with this step-by-step tutorial...",
      "emphasizedKeywords": ["javascript"],
      "siteLinks": [],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    {
      "title": "javascript tutorial",
      "url": "https://www.google.com/search?q=javascript+tutorial&hl=en&gl=us",
      "type": "organic",
      "position": 1
    }
  ],
  "peopleAlsoAsk": [
    {
      "answer": null,
      "question": "What is JavaScript?",
      "title": "What is JavaScript?",
      "url": "https://developer.mozilla.org/...",
      "date": null
    }
    ],
  "customData": {
    "perplexitySearch": {
      "enablePerplexity": false,
      "searchRecency": null,
      "returnImages": false,
      "returnRelatedQuestions": false
    },
    "chatGptSearch": { "enableChatGpt": false },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": "javascript_tutorials_20260416_120101_p1.html"
}

Notes:

• aiOverview appears only if aiMode is not off and serpApiKey is provided.
• html appears only if saveHtml=true; htmlSnapshotUrl appears only if saveHtmlToKeyValueStore=true.
• hasNextPage reflects presence of the “Next” button on the page.

FAQ

Is there a free trial?

Yes. The listing provides 120 trial minutes so you can evaluate the actor before subscribing.

Does it require a proxy?

The actor initializes with Apify’s GOOGLE_SERP proxy by default and automatically falls back to RESIDENTIAL proxies with retries if Google blocks the request. Once residential is used, it continues for subsequent requests.

How many results per query can I get?

Set resultsPerPage (1–100) and maxPagesPerQuery (each page is roughly ~10 organic results). Google may return fewer results on page 1 due to internal filtering.

Can it scrape Google’s AI Overview?

Yes, when you set aiMode (not off) and provide serpApiKey. The aiOverview field will include text and references when available.

How do I extract paid ads and shopping results?

Paid ads appear in paidResults and shopping cards in paidProducts. Enabling focusOnPaidAds improves ad detection by using an ad-specialized proxy with additional retries (extra cost per page).

Does it support mobile vs. desktop SERPs?

Yes. Set mobileResults to true for mobile results; otherwise, desktop results are returned by default.

Can I filter by date or file type?

Yes. Use quickDateRange (e.g., d10, m3, y1) or absolute/relative beforeDate/afterDate. Restrict results to specific fileTypes (e.g., pdf, html, csv).

What export formats are available?

You can export the default dataset as JSON or CSV directly from the Apify UI. The actor pushes standardized SERP JSON per page.

Closing CTA / Final thoughts

The Google Search Results Scraper is built for teams that need structured, accurate SERP data at scale. With robust proxy fallback, granular filters, geo & language targeting, and optional AI Overviews, it delivers analytics-ready outputs for SEO, PPC, research, and engineering workflows. Marketers, developers, analysts, and researchers can run bulk queries, stream results to a dataset, and export JSON/CSV for dashboards or pipelines. Start extracting smarter with a reliable Google SERP scraping API workflow — and turn search intent into actionable insights.