Google Search Results Scraper

Google Search Results Scraper

The Google Search Results Scraper is a production-ready Google SERP scraper that collects structured search data at scale — including organic results, paid ads, paid products, People Also Ask, related queries, optional AI Overview, and more. It’s a Google search results scraper tool for marketers, developers, data analysts, and researchers who need reliable Google SERP data extraction with location, language, and pagination controls. Use it as a real-time Google search scraping API to power SEO monitoring, rank tracking, content research, and competitor analysis at scale.

What is Google Search Results Scraper?

Google Search Results Scraper is a Google SERP data extraction actor that fetches structured search engine result pages (SERPs) from Google and outputs them as clean JSON. It solves the challenge of scalable Google search results crawling by handling pagination, location and language parameters, advanced query filters, and proxy handling automatically. Built for SEO teams, growth marketers, analysts, and developers, it enables large-scale SERP collection, supports bulk queries, and updates datasets in real time for analysis-ready pipelines.

What data / output can you get?

Below are key fields the actor pushes to the dataset (one item per query/page state). Field names reflect the actual JSON keys you’ll get:

Data type	Description	Example value
searchQuery	Metadata about the query and context	{"term": "javascript site:example.com", "url": "https://www.google.com/...", "device": "DESKTOP", "page": 1, "type": "SEARCH", "domain": "google.com", "countryCode": "US", "languageCode": "en", "locationUule": null, "resultsPerPage": 10}
searchQueryTerm	The full query string passed to Google (with applied filters)	"javascript site:example.com"
url	Final SERP URL for the page	"https://www.google.com/search?q=javascript..."
hasNextPage	Indicator for pagination availability in this run	true
serpProviderCode	Internal SERP provider code	"O"
resultsTotal	Parsed “About X results” number when available	1020000
relatedQueries	Top related queries derived from current organic results	[{"title":"JavaScript tutorial","url":"https://www.google.com/search?q=JavaScript+tutorial&hl=en&gl=us"}]
organicResults	Array of organic listings with metadata	[{"title":"MDN Web Docs","url":"https://developer.mozilla.org/...","displayedUrl":"developer.mozilla.org","description":"Resources for developers...","emphasizedKeywords":["javascript"],"siteLinks":[{"title":"Guide","url":"https://developer.mozilla.org/...","description":""}],"productInfo":{},"type":"organic","position":1}]
paidResults	Array of paid ads (if present)	[{"title":"Learn JavaScript Fast","url":"https://example.com/ad","displayedUrl":"example.com/ad","description":"","emphasizedKeywords":["javascript"],"siteLinks":[],"productInfo":{},"type":"paid","position":1}]
paidProducts	Google Shopping-style products (if present)	[{"title":"JavaScript Handbook","displayedUrl":"store.example.com/product","prices":["$19.99"]}]
peopleAlsoAsk	PAA questions with optional URLs	[{"answer":null,"question":"What is JavaScript used for?","title":"What is JavaScript used for?","url":"https://example.com/guide","date":null}]
suggestedResults	Suggested results derived from related queries	[{"title":"JavaScript tutorial","url":"https://www.google.com/search?q=JavaScript+tutorial&hl=en&gl=us","type":"organic","position":1}]
aiOverview	AI overview text and references (if enabled)	{"text":"JavaScript is a programming language...","references":[{"title":"MDN Web Docs","url":"https://developer.mozilla.org/..."}]}
customData	Add-on flags and enrichment settings snapshot	{"perplexitySearch":{"enablePerplexity":false,"searchRecency":null,"returnImages":false,"returnRelatedQuestions":false},"chatGptSearch":{"enableChatGpt":false},"maximumLeadsEnrichmentRecords":0,"leadsEnrichmentDepartments":[]}
htmlSnapshotUrl	Key to saved HTML in key-value store (if enabled)	"javascript_20260415_120301_p1.html"
html	Raw HTML of scraped pages concatenated (if enabled)	"..."

Notes:

• The actor pushes items to the dataset in real time after each page is scraped, making it suitable for streaming pipelines and incremental processing.
• You can view or export results from the Apify dataset in JSON/CSV formats.

Key features

• 🚀 Robust proxy workflow & resilience
  Always uses Apify proxies by default (Google SERP proxy), with automatic fallback to residential proxies on blocks (up to 3 retries) and persistence afterward for consistent success.

• 📦 Rich, SERP-style JSON output
  Returns a structured schema including searchQuery, organicResults, paidResults, paidProducts, peopleAlsoAsk, relatedQueries, suggestedResults, aiOverview (when enabled), totals, and HTML snapshots.

• 🌍 Local & multilingual targeting
  Control country (gl), interface language (hl), search language (lr), and exact location via UULE for a local Google search results crawler workflow.

• 🎯 Advanced query filters
  Apply forceExactMatch, site and relatedToSite, wordsInTitle/inText/inUrl, quickDateRange, beforeDate/afterDate, and fileTypes for precise Google search result parsing.

• 📱 Desktop or mobile SERPs
  Toggle mobileResults to switch between desktop and mobile result layouts for device-specific insights.

• 🧠 AI add-ons (optional, $)
  Enable aiMode with serpApiKey for AI Overview, and compare across engines with Perplexity AI search or ChatGPT search add-ons using customData snapshots.

• 💼 Leads enrichment add-on (optional, $)
  Configure maximumLeadsEnrichmentRecords and leadsEnrichmentDepartments to enrich domains discovered in your SERPs for sales workflows.

• 🖼️ HTML capture & icons
  Save HTML to the dataset (html) or to key-value store (htmlSnapshotUrl) for replay/debugging; includeIcons optionally captures Base64-encoded icon image data.

• 🔗 Developer-friendly & automation-ready
  Works as a Google results scraping software within the Apify platform. Integrate with the Apify API, schedule runs, and connect to pipelines or orchestration tools for a best-in-class Google search results scraper.

How to use Google Search Results Scraper - step by step

1. Create or log in to your Apify account at console.apify.com.
2. Open the Google Search Results Scraper actor.
3. In the Input tab, paste your queries into the “Search term(s)” field (one per line). You can also paste full Google search URLs.
4. Configure scope and targeting:
   • Set resultsPerPage and maxPagesPerQuery (1 page ≈ 10 organic results).
   • Choose countryCode, languageCode, searchLanguage, and optionally locationUule for local targeting.
5. Fine-tune filters and add-ons:
   • Use forceExactMatch, site/relatedToSite, wordsInTitle/wordsInText/wordsInUrl, quickDateRange, beforeDate/afterDate, fileTypes.
   • Enable aiMode with serpApiKey for AI Overview, and optionally Perplexity/ChatGPT add-ons.
   • Toggle mobileResults or includeUnfilteredResults as needed.
6. Configure storage and debug options:
   • saveHtml to dataset (html), and/or saveHtmlToKeyValueStore for browser-viewable snapshots (htmlSnapshotUrl).
7. Set Proxy configuration (default uses Google SERP proxy automatically; residential fallback is handled internally on blocks).
8. Click Start to run the actor and watch the Log for page-by-page progress and any proxy fallback messages.
9. Export data from the Output tab as JSON/CSV for analysis or integration.

Pro Tip: Use this actor as a real-time Google search results JSON scraper in an automated pipeline by triggering runs via the Apify API and streaming dataset items into your analytics stack.

Use cases

Use case name	Description
SEO rank tracking & SERP monitoring	Collect organicResults and resultsTotal across locations and languages to monitor rankings and visibility over time.
Competitor & keyword research	Aggregate relatedQueries, peopleAlsoAsk, and organicResults to uncover content gaps and competitor strategies.
Google Ads & shopping insights	Enable focusOnPaidAds and capture paidResults and paidProducts for ad intelligence and product pricing snapshots.
Local SERP audits	Apply countryCode, languageCode, searchLanguage, and locationUule for a local Google search results scraper workflow.
Content planning & AEO/GEO	Use aiMode (with serpApiKey) to capture AI Overview text and references, informing Answer/Generative Engine Optimization.
Bulk SERP data pipelines	Ingest hundreds/thousands of queries (one per line) for a bulk Google search results scraper feeding BI dashboards.
Academic & market research	Export JSON/CSV datasets for reproducible studies of query intent, PAA trends, and SERP composition.
Enrichment & lead workflows	Use maximumLeadsEnrichmentRecords and leadsEnrichmentDepartments to augment discovered domains with business contacts.

Why choose Google Search Results Scraper?

The scraper is engineered for precision, automation, and scalability—an ideal Google SERP scraping service alternative without browser flakiness.

• 🎯 Accurate SERP parsing with structured fields for organic, paid, shopping, PAA, and related queries
• 🌐 Localized targeting via country, interface/search language, and exact location (UULE)
• 📈 Built for scale with bulk inputs, pagination, and resilient proxy fallback (SERP → residential)
• 🧩 Developer-ready JSON schema and real-time dataset updates for downstream pipelines
• 🛡️ Public-page only design with optional HTML snapshots for auditability
• 💸 Clear add-on controls for AI and leads enrichment when needed
• 🔁 More reliable than extension-based tools; production-ready infrastructure on Apify

In short: a reliable, real-time, and structured Google search results scraper built for teams that need trustworthy data at scale.

Is it legal / ethical to use Google Search Results Scraper?

Yes — when used responsibly. This actor collects data from publicly available Google Search result pages and does not access private or authenticated content.

Recommended guidelines:

• Scrape only public SERP data and comply with Google’s Terms of Service and robots.txt.
• Adhere to data protection laws (e.g., GDPR/CCPA) and avoid scraping personal data without a legitimate basis.
• Use rate limits and proxy handling responsibly to minimize impact and avoid abuse.
• Consult your legal team for edge cases or jurisdiction-specific rules.

Input parameters & output format

Example JSON input

{
  "queries": "javascript\ntypescript\npython",
  "resultsPerPage": 10,
  "maxPagesPerQuery": 2,
  "countryCode": "us",
  "languageCode": "en",
  "saveHtmlToKeyValueStore": true,
  "aiMode": "aiModeOff",
  "mobileResults": false
}

All input fields

• queries (string, required): Use regular search words or enter Google Search URLs. Advanced Google operators supported; keep queries ≤ 32 words.
• resultsPerPage (integer, default: 100, min: 1, max: 100): Desired results per page (Google may return fewer, especially on page 1). Consider includeUnfilteredResults for more results.
• maxPagesPerQuery (integer, default: 1, min: 1): Maximum pages to scrape per query (≈10 results per page).
• aiMode (string, default: "aiModeOff", enum: ["aiModeOff", "aiModeWithSearchResults", "aiModeOnly"]): Enable scraping of Google’s AI Mode for AEO/GEO and competitor tracking.
• serpApiKey (string, nullable, secret): Optional; when set and aiMode is not off, fetches AI Overview via SerpApi.
• perplexitySearch (object): Perplexity AI search add-on settings (additional fees may apply).
  • enablePerplexity (boolean, default: false)
  • searchRecency (string, nullable, enum: "", "day", "week", "month", "year")
  • returnImages (boolean, default: false)
  • returnRelatedQuestions (boolean, default: false)
• chatGptSearch (object): ChatGPT Search add-on settings (additional fees may apply).
  • enableChatGpt (boolean, default: false)
• maximumLeadsEnrichmentRecords (integer, default: 0, min: 0): Maximum leads to enrich per domain (optional add-on; multiplier cost warning).
• leadsEnrichmentDepartments (array of strings, default: []): Filter leads by departments (e.g., "marketing", "sales"). Works only if maximumLeadsEnrichmentRecords > 0.
• focusOnPaidAds (boolean, default: false): Enable paid results (ads) extraction with specialized retries for ad detection.
• countryCode (string, default: "us"): Country for the search and domain mapping (e.g., google.es for Spain).
• searchLanguage (string, nullable): Restrict results to a language (lr parameter), e.g., "de" for German.
• languageCode (string, default: "en"): Interface language (hl parameter).
• locationUule (string, nullable): Exact location (UULE parameter) for local SERPs.
• forceExactMatch (boolean, default: false): Wrap queries in quotes for exact match (may return fewer results).
• site (string, nullable): Limit to a specific site, e.g., site:example.com.
• relatedToSite (string, nullable): Pages related to a site, e.g., related:example.com (ignored if site is set).
• wordsInTitle (array of strings, default: []): Filter by words appearing in titles (intitle: operator).
• wordsInText (array of strings, default: []): Filter by words in body text (intext: operator).
• wordsInUrl (array of strings, default: []): Filter by words in URLs (inurl: operator).
• quickDateRange (string, nullable): Quick date ranges using qdr (e.g., d10, w1, m3, y1).
• beforeDate (string, nullable, absoluteOrRelative): Filter before date (YYYY-MM-DD) or relative (e.g., "8 days", "3 months").
• afterDate (string, nullable, absoluteOrRelative): Filter after date (YYYY-MM-DD) or relative.
• fileTypes (array of strings, default: []): Filter by file types (e.g., "pdf", "csv", "docx", "ipynb", "sql", "xml", etc.).
• mobileResults (boolean, default: false): Return mobile SERPs if checked; otherwise desktop results.
• includeUnfilteredResults (boolean, default: false): Include lower-quality results that Google normally filters out.
• saveHtml (boolean, default: false): Save SERP HTML to the dataset (html field).
• saveHtmlToKeyValueStore (boolean, default: true): Save SERP HTML to key-value store and expose link in htmlSnapshotUrl.
• includeIcons (boolean, default: false): Include Base64-encoded icon image data in result objects when found.
• proxyConfiguration (object, editor: proxy): Proxy settings. By default, uses Google SERP proxy and falls back to residential (3 retries) if blocked; residential sticks for the rest of the run.

Example JSON output

{
  "searchQuery": {
    "term": "javascript site:developer.mozilla.org",
    "url": "https://www.google.com/search?q=javascript+site%3Adeveloper.mozilla.org&hl=en&gl=us",
    "device": "DESKTOP",
    "page": 1,
    "type": "SEARCH",
    "domain": "google.com",
    "countryCode": "US",
    "languageCode": "en",
    "locationUule": null,
    "resultsPerPage": 10
  },
  "searchQueryTerm": "javascript site:developer.mozilla.org",
  "url": "https://www.google.com/search?q=javascript+site%3Adeveloper.mozilla.org&hl=en&gl=us",
  "hasNextPage": true,
  "serpProviderCode": "O",
  "resultsTotal": 1020000,
  "relatedQueries": [
    {
      "title": "JavaScript tutorial",
      "url": "https://www.google.com/search?q=JavaScript+tutorial&hl=en&gl=us"
    }
  ],
  "paidResults": [
    {
      "title": "Learn JavaScript Fast",
      "url": "https://example.com/ad",
      "displayedUrl": "example.com/ad",
      "description": "",
      "emphasizedKeywords": ["javascript"],
      "siteLinks": [],
      "productInfo": {},
      "type": "paid",
      "position": 1
    }
  ],
  "paidProducts": [
    {
      "title": "JavaScript Handbook",
      "displayedUrl": "store.example.com/product",
      "prices": ["$19.99"]
    }
  ],
  "aiOverview": null,
  "organicResults": [
    {
      "title": "JavaScript | MDN",
      "url": "https://developer.mozilla.org/en-US/docs/Web/JavaScript",
      "displayedUrl": "developer.mozilla.org/en-US/docs/Web/JavaScript",
      "description": "JavaScript (JS) is a lightweight, interpreted....",
      "emphasizedKeywords": ["javascript"],
      "siteLinks": [
        {
          "title": "Guide",
          "url": "https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide",
          "description": ""
        }
      ],
      "productInfo": {},
      "type": "organic",
      "position": 1
    }
  ],
  "suggestedResults": [
    {
      "title": "JavaScript tutorial",
      "url": "https://www.google.com/search?q=JavaScript+tutorial&hl=en&gl=us",
      "type": "organic",
      "position": 1
    }
  ],
  "peopleAlsoAsk": [
    {
      "answer": null,
      "question": "What is JavaScript used for?",
      "title": "What is JavaScript used for?",
      "url": "https://example.com/guide",
      "date": null
    }
  ],
  "customData": {
    "perplexitySearch": {
      "enablePerplexity": false,
      "searchRecency": null,
      "returnImages": false,
      "returnRelatedQuestions": false
    },
    "chatGptSearch": {
      "enableChatGpt": false
    },
    "maximumLeadsEnrichmentRecords": 0,
    "leadsEnrichmentDepartments": []
  },
  "htmlSnapshotUrl": "javascript_20260415_120301_p1.html"
}

Fields that may be null or absent depending on settings/results:

• aiOverview (requires aiMode enabled and serpApiKey provided)
• paidResults / paidProducts (only if present for the query)
• html (only when saveHtml is true)
• htmlSnapshotUrl (only when saveHtmlToKeyValueStore is true)

FAQ

Does this support bulk Google search results scraping?

Yes. Add multiple queries (or full Google search URLs) in the “Search term(s)” field—one per line—and the actor will process them in sequence, pushing results to the dataset in real time.

How many Google results can I get per query?

Google typically shows about 10 organic results per page. Set resultsPerPage (1–100) and increase maxPagesPerQuery to collect more (e.g., 10 pages ≈ ~100 organic results, depending on Google’s actual returns).

Can I target specific countries, languages, or exact locations?

Yes. Use countryCode and languageCode for interface targeting, searchLanguage to restrict result language, and locationUule for exact location (UULE) to build a local Google search results scraper configuration.

Does it scrape Google Ads and Shopping results?

It can. Enable focusOnPaidAds to improve ad detection (with specialized retries) and the actor will return paidResults and paidProducts when they appear for your queries.

How do I capture AI Overview from Google?

Set aiMode to a non-off state and provide serpApiKey. When both are set, the actor fetches aiOverview, returning summary text and references when available.

Can I get mobile SERPs?

Yes. Set mobileResults to true to fetch results tailored to the mobile layout; otherwise desktop results are returned.

What if Google blocks my requests?

The actor initializes with the Google SERP proxy and automatically falls back to residential proxies with up to 3 retries when blocks are detected. Once residential is used, it stays in use for the remainder of the run.

Which formats can I export?

From the Apify dataset, you can export to JSON/CSV directly from the Output tab, suitable for downstream analysis and ingestion into your tools.

Closing CTA / Final thoughts

Google Search Results Scraper is built for accurate, scalable SERP extraction with real-time, structured JSON output. With granular location/language targeting, advanced filters, optional AI and leads enrichment add-ons, and resilient proxy handling, it’s the best Google search results scraper for SEO, marketing, analytics, and research teams. Developers can trigger runs via the Apify platform and stream datasets into pipelines for automation. Start extracting smarter Google SERP insights today with a reliable, real-time Google SERP scraper.