Google Search Scraper: SERP API with AI Overview, Ads, PAA

From The Mine Works, makers of Threads Scraper and B2B Leads Finder, with over 245,000 runs across 280+ public actors.

Updated 27 Oct 2026: first release. Queries or Google search links in; per results page the organic results with their real links, the ads, People Also Ask, related searches, the AI Overview text and the knowledge panel, as one row per page or one row per result. Bookmark this Actor on its Store page so it is one click away next time.

Why choose this actor?

• Real result links, not Google redirects. Since August 2026 Google hides most result links behind an opaque /goto token. This actor follows each one once, so url is the page itself: 176 of 176 organic results in our 20 query test came back with their real address, plus domain and the displayed_url line for every row.
• The whole page in one row, or one row per result. Organic results with position, title, snippet, date and sitelinks, the ads with their landing pages, People Also Ask, related searches, the AI Overview with its sources and the knowledge panel (type, description, founders, CEO, headquarters, social profiles). Switch outputFormat to results for a flat table ready for a rank tracker or a spreadsheet.
• Searches as a visitor in your country. Each query is sent with your country as Google's location, so results are not mixed with the proxy's own country. Pay per results page read; retries, blocked pages, pages with no results and the summary rows are never charged.

Part of The Mine Works Marketing, SEO and reviews family: Facebook Ad Library Scraper, Google Ads Transparency Scraper, Similarweb Scraper, Google News Scraper, Trustpilot Reviews Scraper, Google Trends Scraper.

Try it in one minute

Paste this into the JSON tab of the input page and press Start. It reads the first results page for "crm software" in the United States. In our test it took 15 seconds from start to finish.

{
  "queries": ["crm software"]
}

Put more queries on separate lines to run them side by side, raise maxPagesPerQuery to go deeper than the top 10, or paste a Google search link from your browser (its filters, such as the date range, are kept).

Apify's free plan includes $5 of credit every month, which covers about 1,100 results pages at this actor's price.

Copy to your AI assistant

themineworks/google-search-scraper on Apify. Reads Google Search results pages (no login) and returns, per page: organic_results (position, title, url, url_resolved, displayed_url, domain, site_name, meta, snippet, emphasized, date, date_iso, sitelinks), paid_results (ad_position, block, title, url, displayed_url, domain, advertiser, description), people_also_ask (question), related_searches (query, url), ai_overview (text, sources with title, site, snippet, url), knowledge_panel (title, type, description, description_source, description_url, website, facts, profiles), total_results, has_next_page, plus query, page, search_url, country_code, language_code, google_domain, location, scraped_at. Call ApifyClient("TOKEN").actor("themineworks/google-search-scraper").call(run_input={"queries": ["crm software"]}), then client.dataset(run["defaultDatasetId"]).list_items().items. Input: queries (list of search terms or google.<tld>/search?q= links, up to 1000), countryCode (default "us"), languageCode (default "en"), maxPagesPerQuery (1 to 20, default 1), resultsPerPage (1 to 10, default 10), outputFormat ("pages" one row per results page, default; "results" one flat row per organic result), resolveUrls (default true), googleDomain (default "google.com"), location (optional Google location name such as "Austin,Texas,United States", or a uule code). Billing is per results page delivered. Rows with a _type field ("summary", "info", "error") are never charged; skip them. Full spec: GET https://api.apify.com/v2/acts/themineworks~google-search-scraper/builds/default (Bearer TOKEN). Token: https://console.apify.com/account/integrations?fpr=ymnoit&utm_source=apify-readme&utm_medium=referral

Key features

• Queries or links. Type queries as you would in Google, operators included (site:, intitle:, "exact phrase", -word, OR). Or paste a Google search link from your browser: its own filters (tbs date range, lr, cr, safe, tbm), country and language are kept, and its start sets the first page read.
• Pages, not guesses. Google serves 10 organic results a page and has ignored larger num values since September 2025, so depth comes from maxPagesPerQuery. Reading stops early when Google shows no next page.
• Every block of the page. Ads at the top, between results and at the bottom (block), People Also Ask questions, related searches, the AI Overview text and its sources, and the knowledge panel facts for companies and brands.
• Country, language and city. countryCode sets the ranking country (gl), languageCode the interface language (hl), location a city or region such as Austin,Texas,United States.
• Fast single calls. One query is one results page request plus a second or two of link checks, at 256 MB: 15 seconds in our test. Large lists run 5 queries side by side.
• Stops cleanly. A page that has not started to answer after 15 seconds gets a second request on a new IP, and the first answer is kept. A refused page or a captcha is retried on a new IP, up to 3 times. Five failed pages in a row end the run; the run stops before its time limit and keeps every row. The summary row says why.

How to use it

Rank tracking for a keyword list

{
  "queries": ["crm software", "sales engagement platform", "revenue intelligence"],
  "maxPagesPerQuery": 3,
  "outputFormat": "results",
  "countryCode": "us"
}

One row per organic result with position counted across the pages, so the top 30 of each keyword is a flat table. Filter on domain to see where your site and each competitor rank, and schedule the run daily to track changes.

Competitor ads and landing pages

{
  "queries": ["microsoft dynamics 365 partner", "erp consultants"],
  "countryCode": "us"
}

paid_results lists each ad with its title, advertiser, landing page and text, and block says whether it ran at the top, between results or at the bottom. In our test "top seo agencies new york" showed 6 ads on one page.

Content research from questions and AI Overviews

people_also_ask, related_searches and ai_overview show what Google answers for a topic and which sources it cites. In our 20 query test 14 pages carried an AI Overview and 18 a People Also Ask block.

Company facts and lead lists

A company name returns the knowledge panel: type, description, founders, CEO, headquarters and social profiles. site: queries such as site:linkedin.com/company fintech london return one LinkedIn company page per result, with its follower line in meta. Google sometimes ignores site: and answers with other sites; the actor then reads the page once more, and if Google ignores it again the row says site_operator_ignored: true (one charge either way).

Another country or a city

{
  "queries": ["steuerberater software"],
  "countryCode": "de",
  "languageCode": "de",
  "googleDomain": "google.de",
  "location": "Munich,Bavaria,Germany"
}

Input parameters

Parameter	Type	Default	What it does
queries	array of strings	required	Search terms or Google search links, one per line, up to 1,000.
countryCode	string	us	Country the results are ranked for (gl).
languageCode	string	en	Google interface language (hl).
maxPagesPerQuery	integer	1	Results pages per query, 1 to 20. Page 2 is results 11 to 20.
resultsPerPage	integer	10	Keep only the top N organic results of each page, 1 to 10. Same price per page.
outputFormat	string	pages	pages: one row per results page with every block. results: one flat row per organic result.
resolveUrls	boolean	true	Follow Google's link redirects so url holds the real address. Off is faster; domain and displayed_url stay filled.
googleDomain	string	google.com	Google site to search, for example google.co.uk.
location	string	empty	City or region as a Google location name, or a uule code. Empty means the whole country of countryCode.

What data do you get?

With outputFormat: "pages", one row per results page. Rows with a _type field are the summary, info and error rows; filter them out on _type.

Field	What it is
query, page, search_url	The query, the results page number (1 is the first) and the Google page read
country_code, language_code, google_domain, location	Where and how Google was asked
organic_count, organic_results	The organic results of the page (fields below)
paid_count, paid_results	The ads: ad_position, block (top, inline, bottom), title, url (landing page), displayed_url, domain, advertiser, description
people_also_ask	People Also Ask questions, question each
related_searches	Related searches: query and a Google url for it
ai_overview	When Google shows one: text (as shown before "Show more") and sources (title, site, snippet, url)
knowledge_panel	When Google shows one: title, type, description, description_source, description_url, website, facts (CEO, founders, headquarters and so on), profiles (social links)
total_results, total_results_raw	The result count Google states, and the line as shown
has_next_page	Google shows a next page for the query
site_operator_ignored	true when the query has site: and, on two reads, fewer than half of the results were on that site
scraped_at	When the row was written

Each organic result (and, with outputFormat: "results", each row, alongside query, page, search_url, country_code, language_code, google_domain and location):

Field	What it is
position	Rank across the pages read, 1 is the top result
title, url	Result title and the page it links to
url_resolved	true when url holds the real address
displayed_url	The address line under the title, e.g. https://www.g2.com › Sales Tools
domain	Host of the result, e.g. g2.com
site_name	Site name above the title, e.g. G2 or LinkedIn · HubSpot
meta	The line shown instead of an address on social results, e.g. 28.9K+ followers
snippet, emphasized	The description, and the words Google set in bold
date, date_iso	Date shown before the snippet, as shown and as YYYY-MM-DD
sitelinks	Links under the result: title, url, snippet

Summary row (_type: "summary", never charged): queries_requested, queries_read, queries_failed, pages_requested, pages_delivered, charged_pages, organic_results_delivered, ads_found, ai_overviews_found, knowledge_panels_found, urls_resolved, urls_unresolved, serp_requests, serp_requests_no_answer, serp_walls, avg_page_seconds, stopped_by_max_cost, stopped_by_block_guard, ended_on_deadline, shortfall_reason when fewer pages came back than asked, and per_query, one entry per query with pages, organic_results and why it stopped. The same summary is saved as the OUTPUT record of the run.

Error rows (_type: "error", never charged): one per input that is not a query or a Google search link (invalid_input), and one per query whose first page could not be read after 3 tries (blocked), with identifier and reason.

Stable fields for automations

Present in every page row of our test runs, and their names will not change: query, page, search_url, country_code, language_code, google_domain, organic_count, organic_results, paid_count, paid_results, has_next_page, scraped_at. In every organic result: position, title, url, url_resolved, domain, snippet.

Optional, present only when Google shows them: ai_overview (14 of 20 test pages), people_also_ask (18 of 20), related_searches (17 of 20), knowledge_panel (company and brand queries), date (80 of 176 results), sitelinks (brand and homepage results), meta (social results), displayed_url (not on social results), total_results.

Output examples

One page row from the 20 query test (27 Oct 2026, trimmed to two organic results):

{
  "query": "what is account based marketing",
  "page": 1,
  "search_url": "https://www.google.com/search?q=what+is+account+based+marketing&hl=en&gl=us&uule=w%2BCAIQICINVW5pdGVkIFN0YXRlcw%3D%3D",
  "country_code": "us",
  "language_code": "en",
  "google_domain": "google.com",
  "location": "United States",
  "organic_count": 8,
  "organic_results": [
    {
      "position": 1,
      "title": "Account Based Marketing: the Complete Guide",
      "url": "https://www.leadfeeder.com/blog/marketing-strategy/what-is-account-based-marketing/",
      "url_resolved": true,
      "displayed_url": "https://www.leadfeeder.com › Blog › Marketing Strategy",
      "domain": "leadfeeder.com",
      "site_name": "Leadfeeder",
      "snippet": "Account-based marketing (ABM) is a B2B marketing strategy that focuses on targeting high-value accounts with highly personalized campaigns ...",
      "emphasized": ["a B2B marketing strategy that"],
      "date": "Sep 14, 2026",
      "date_iso": "2026-09-14",
      "sitelinks": []
    }
  ],
  "paid_count": 0,
  "paid_results": [],
  "people_also_ask": [
    { "question": "What are the 5 C's of marketing strategy?" },
    { "question": "What do marketing account managers do?" }
  ],
  "related_searches": [
    { "query": "Account-based marketing tools", "url": "https://www.google.com/search?q=Account-based%20marketing%20tools" }
  ],
  "ai_overview": {
    "text": "Account-Based Marketing (ABM) is a business-to-business (B2B) marketing strategy where companies focus their resources on a carefully selected set of high-value accounts, treating each individual account as a distinct \"market of one\" rather than casting a wide net for general leads.\nCore Approaches of ABM\n* One-to-One (Strategic ABM): Deeply customized, bespoke campaigns built for single, high-value enterprise accounts.",
    "sources": [
      { "title": "Account-Based Marketing (ABM) - Salesforce", "site": "Salesforce", "url": "https://www.salesforce.com/marketing/account-based-marketing-guide/" }
    ]
  },
  "has_next_page": true,
  "scraped_at": "2026-10-27T11:22:33.350Z"
}

The knowledge panel and an ad from the same run (trimmed):

{
  "query": "snowflake data cloud",
  "knowledge_panel": {
    "title": "Snowflake",
    "type": "Computer software company",
    "description": "Snowflake Inc. is an American cloud-based artificial intelligence data platform company founded in San Mateo, California, and headquartered in Menlo Park.",
    "description_source": "Wikipedia",
    "description_url": "https://en.wikipedia.org/wiki/Snowflake_Inc.",
    "facts": {
      "Headquarters": "Menlo Park, CA",
      "Founders": "Benoît Dageville, Thierry Cruanes",
      "Founded": "July 23, 2012, San Mateo, CA"
    },
    "profiles": [
      { "name": "LinkedIn", "url": "https://www.linkedin.com/company/snowflake-computing" },
      { "name": "YouTube", "url": "https://www.youtube.com/user/snowflakecomputing" }
    ]
  }
}

{
  "query": "top seo agencies new york",
  "paid_results": [
    {
      "ad_position": 1,
      "block": "top",
      "title": "List of Top SEO Firms - Top SEO Companies 2026",
      "url": "https://www.goodfirms.co/",
      "displayed_url": "https://www.goodfirms.co › seo-agencies",
      "domain": "goodfirms.co",
      "advertiser": "Goodfirms",
      "description": "Browse 500+ verified SEO companies on Goodfirms. Filter by budget, reviews & niche now. Choose top-ranked SEO agencies."
    }
  ]
}

One row with outputFormat: "results" (20 query test in results mode):

{
  "query": "crm software",
  "page": 1,
  "search_url": "https://www.google.com/search?q=crm+software&hl=en&gl=us&uule=w%2BCAIQICINVW5pdGVkIFN0YXRlcw%3D%3D",
  "country_code": "us",
  "language_code": "en",
  "google_domain": "google.com",
  "location": "United States",
  "position": 2,
  "title": "Best CRM Software of 2026",
  "url": "https://www.businessnewsdaily.com/7839-best-crm-software.html",
  "url_resolved": true,
  "displayed_url": "https://www.businessnewsdaily.com › 7839-best-crm-so...",
  "domain": "businessnewsdaily.com",
  "site_name": "Business News Daily",
  "snippet": "Compare the best CRM software for small business, from free plans to AI agents that handle outreach, follow-ups and reporting for you.",
  "emphasized": ["best CRM software for small business"],
  "date": "Sep 8, 2026",
  "date_iso": "2026-09-08",
  "sitelinks": [],
  "scraped_at": "2026-10-27T11:24:56.911Z"
}

The summary row of that run, trimmed:

{
  "_type": "summary",
  "queries_requested": 20,
  "queries_read": 20,
  "pages_delivered": 20,
  "charged_pages": 20,
  "organic_results_delivered": 176,
  "ads_found": 17,
  "ai_overviews_found": 14,
  "urls_resolved": 176,
  "urls_unresolved": 0,
  "serp_requests": 21,
  "serp_requests_no_answer": 1,
  "avg_page_seconds": 6.7,
  "run_seconds": 39
}

An error row for a link that is not a Google search (nothing charged):

{ "_type": "error", "error_type": "invalid_input", "identifier": "https://example.com/not-google", "reason": "Not a Google search link. Use a link like https://www.google.com/search?q=crm+software, or type the words as a query." }

Pricing

Pay per event: you pay for the results pages you receive, plus a flat $0.0005 per run, whatever memory you choose. Resolving links, the AI Overview, the knowledge panel and the one row per result format cost nothing extra.

Event	When it is charged	Free plan	Bronze (Starter)	Silver (Scale)	Gold (Business)
search-page	Once per Google results page delivered to your dataset	$4.40 per 1,000	$4.40 per 1,000	$4.40 per 1,000	$4.40 per 1,000
run-start	Once per run, after the first results page is delivered	$0.0005	$0.0005	$0.0005	$0.0005

A page is charged once whether it lands as one page row or as up to 10 result rows. The price of one page is the price per 1,000 divided by 1,000.

Never charged:

• retries, pages that did not answer, captcha or empty pages Google served instead of results;
• queries with no results, and pages past the last one;
• inputs that are not queries or Google search links (each gets an error row);
• the summary, info and error rows;
• a run whose queries all fail: the start fee is charged only once a page is delivered;
• anything past your "Max cost per run". The actor reserves the budget for a page before it requests it, so it never reads a page it cannot deliver, and it stops while the total is still below your limit: the run ends SUCCEEDED with a partial result and a status message saying so.

Run it on a schedule

1. Open the actor in Apify Console, set your input and click Save as a new task.
2. In the task, open Schedules, click Add schedule and pick how often (every day, every Monday at 08:00 and so on).
3. With outputFormat: "results", compare runs on query, domain and position to see who moved up or down.
4. Optional: add an integration (Google Sheets, Slack, a webhook) so each run lands where your team works.

FAQ

What does this actor read?

Google's public results pages, as a visitor who is not signed in sees them, through Apify's Google SERP proxy. With resolveUrls on, it also checks each result's redirect once to read the real address. It never clicks ads: an ad's landing page comes from the page itself.

How many results do I get per page?

Up to 10 organic results, which is what Google serves. Brand queries often show fewer web results because Google fills the page with the brand's social profiles and a knowledge panel. Google has ignored num=20 to num=100 since September 2025, so use maxPagesPerQuery for depth: 3 pages is about the top 30.

Why are some links checked one by one?

On most pages Google no longer puts the result's address in the link; it puts an opaque token that only Google can turn into the address. The actor asks Google for each token's target once (about 1 second, 8 at a time). Turn resolveUrls off if you only need domain and displayed_url; url is then filled only where Google shows it.

Can I get mobile results?

Not with this actor. The Google SERP proxy answers with the desktop page whatever device it is asked for (we tested Android and iPhone browsers), so every row is the desktop result.

Which country are the results from?

The one you set in countryCode. The proxy's own IP can be in any country, so the actor also sends that country to Google as the searcher's location. With it, a "weather" search for the United States showed a US location, and 0 of 36 ads in two 20 query runs came from another country; without it, Google said it could not determine a location and 1 ad in 13 was a Japanese one.

Why is the AI Overview text short?

It is the text Google prints before its "Show more" button; the rest loads only when a person clicks. It is usually 1 to 4 paragraphs or a short list, with the sources Google cites.

Why do some site: queries return other sites?

Google drops the site: operator now and then. In our tests the same site:linkedin.com/company query returned LinkedIn pages on 2 runs out of 3 and other sites on the third, and a site:linkedin.com/in query came back with 1 LinkedIn result in 10 on both reads. When that happens the actor reads the page again on a new IP, at its own cost, and flags the row if Google drops the operator twice.

Is total_results reliable?

No. Google now shows its result count only in a hidden menu, and the number it gives changes from one request to the next (101, 124 and 251,000 for the same query on the same day). It is returned as Google states it; do not use it to compare queries.

How long does a run take?

In our tests: 15 seconds for one query, 40 seconds for 20 queries of one page each, 22 seconds for 2 queries of 2 pages each. Google starts to answer after about 5 seconds (9 seconds for the slowest tenth). About 1 request in 20 gets no answer at all; after 15 seconds a second request is sent on a new IP, so such a page takes about 25 seconds instead of failing.

Do I need a proxy or a Google account?

No. The actor uses Apify's Google SERP proxy, which is included in the price, and no Google account.

Which formats can I download?

JSON, CSV, Excel, XML, RSS and HTML from the run's Storage tab or through the API. The dataset has ready views: Pages, Organic results, Ads and One row per result. In CSV and Excel nested fields are flattened into columns such as organic_results/0/url; use outputFormat: "results" for a clean table.

Can I use it from Claude, ChatGPT or Cursor (MCP)?

Yes, through Apify's MCP server. Connector URL: https://mcp.apify.com/?tools=themineworks/google-search-scraper

• Claude: Settings > Connectors > Add custom connector, paste the URL, sign in with Apify.
• ChatGPT: turn on developer mode, add an MCP connector with the URL, sign in with Apify.
• Cursor or VS Code: add it as an HTTP MCP server with that URL.
• Claude Code: claude mcp add -t http google-search-scraper "https://mcp.apify.com/?tools=themineworks/google-search-scraper"

Is it legal to scrape Google?

The actor reads only public results pages that anyone can see without logging in. You are responsible for how you use the data, including Google's terms and data protection law such as GDPR and CCPA when results name people. This is general information, not legal advice.

Integrations

• Google Sheets: send each run's rows to a sheet with Apify's Google Sheets integration; outputFormat: "results" fits a sheet as is.
• Make, Zapier and n8n: start a run and read its dataset with the Apify apps and nodes, for example "our rank for 50 keywords every Monday to Slack".
• Clay and other enrichment tools: call the actor with one query per company and read knowledge_panel and the top results.
• Webhooks: have Apify call your URL when a run succeeds, then fetch the dataset.
• API and SDKs: run it from Python or JavaScript with the Apify client, or with a plain HTTP call (see "Copy to your AI assistant" above).

More from The Mine Works

Marketing, SEO and reviews

• Facebook Ad Library Scraper
• Google Ads Transparency Scraper
• Similarweb Scraper
• Google News Scraper
• Trustpilot Reviews Scraper
• Google Trends Scraper
• Semrush Scraper
• App Reviews Scraper
• Trustpilot Business Search
• TripAdvisor Reviews Scraper
• Username Checker
• Backlink Building Agent

Social media and video

• Threads Scraper
• Reddit Scraper
• Threads Search Scraper
• Instagram Profile Scraper

Leads and business directories

• B2B Leads Finder
• Skip Trace Lookup
• Google Maps Email Scraper
• 2GIS Places Scraper

LinkedIn

• LinkedIn Company Scraper
• LinkedIn Post Scraper
• LinkedIn Employees Scraper
• LinkedIn Profile Scraper

Real estate

• Zillow Rentals Scraper
• Zillow Sold Comps Scraper
• India Real Estate MCP
• Housing.com Scraper

Science, health and government data

• Academic Research MCP
• CourtListener Scraper
• Socrata Open Data Scraper
• OpenAlex Scraper

Jobs and hiring

• Foundit Monster India Jobs
• Hirist Jobs Scraper
• India Jobs MCP
• Naukri Jobs Scraper

Company and business data

• Company Domain Finder
• GST Taxpayer Lookup
• Company KYB Resolver
• SEC EDGAR Filings Scraper

E-commerce and marketplaces

• Ozon.ru Scraper
• ⭐ Amazon Reviews Scraper
• Amazon Product Scraper
• Etsy Listings Scraper

Food and local services

• NoBroker Scraper
• Swiggy Restaurant Scraper
• Zomato Scraper

Developer and AI tools

• Website to Markdown Crawler
• GitHub Skill Finder
• GitHub Repo Scraper
• GitHub Trending Scraper

More tools

• G2 Reviews Scraper
• Naver Blog Scraper
• WeChat Article Scraper 微信公众号文章
• Taobao Products Scraper 淘宝 天猫

Support

Found a query that returns fewer results than Google shows you, or a field that comes back empty? Open an issue in the Issues tab with the input and the run ID, and we will look at it. For a field you need or a new source, email dmineworks@gmail.com.

Search Google by query or link and get each results page back as data: organic results with real links, ads, People Also Ask, related searches, the AI Overview and the knowledge panel, one row per page or per result, no login.