Google Search Results Scraper

🔎 What is a Google scraper?

Our free SERP scraper crawls Google Search Results Pages (SERPs) and extracts data from those web pages in structured formats such as JSON, XML, CSV, or Excel. With this SERP Scraper API, you will be able to extract the following Google data from each Google page:

🌱 Organic results	🛍 Paid results
📢 Product ads	❓ Related queries
🙋‍♀️ People Also Ask	🏷 Prices
⭐️ Review rating and review count	🔍 Additional custom attributes

💯 How many results can you scrape with Google Search Scraper?

Google Search Scraper can return up to 300 results for one keyword. This is due to the limitations of Google itself: although it shows as if it has millions of results for a given search query, Google will always display a maximum of three pages. With a maximum 100 results shown per page, this makes up to 300 results in total.

If your use case doesn't require otherwise, we strongly recommend you to keep the setting or Results per Google page to the max 100, so that you get 100 results per 1 page instead of crawling 10 pages, each with 10 results. The 10 by 10 is also possible but since you’re paying for request, it will be ten times more pricey than it could’ve been).

While we regularly run Actor tests to keep the benchmarks in check, the results may also fluctuate without our knowing. The best way to know for sure for your particular use case is to do a test run yourself.

💸 How much will scraping Google Search cost you?

When it comes to scraping, it can be challenging to estimate the resources needed to extract data as use cases may vary significantly. That's why the best course of action is to run a test scrape with a small sample of input data and limited output. You’ll get your price per scrape, which you’ll then multiply by the number of scrapes you intend to do.

Watch this video for a few helpful tips. And don't forget that choosing a higher plan will save you money for using this Google SERP API in the long run.

⬇️ Google Search data input

The scraper gives you really good control over what kind of Google Search results you'll get. You can specify the following settings:

• Query phrases or raw Google search URLs 🔍
• Country/search domain 🗺
• Language of search 🇬🇧
• Exact geolocation 📍
• Number of results per page 🔟
• Mobile or desktop version results 📱

For a complete description of all settings, see the Input tab.

How to scrape Google Search results by URL or keyword

There are two ways you can scrape Google search pages: either by URL or by search term.

1. Scraping by URL will get you data from Google Search results page. You'll get Google data from a copy-pasted Google URL with any Google country domain (e.g. google.co.uk). You can add as many URLs as you want.
2. Scraping by search term will also get you data from Google Search results page. You can also add as many search terms as you want.

Example input for scraping Google Search search term

It is also easy to get Google search data by search term. Just enter the search term and a number of Google pages to scrape. With this option, you also can:

• scrape by multiple keywords in parallel by adding more search terms and separating them by a new line
• indicate how many results you want to see per each Google page (10-100)
• indicate the country of search (domain), language, and UULE location parameter

Here's its equivalent in JSON:

1{
2  "countryCode": "us",
3  "customDataFunction": "async ({ input, $, request, response, html }) => {\\\\n  return {\\\\n    pageTitle: $('title').text(),\\\\n  };\\\\n};",
4  "includeUnfilteredResults": false,
5  "languageCode": "en",
6  "maxPagesPerQuery": 2,
7  "mobileResults": false,
8  "queries": "hotels in Seattle \\n hotels in New York",
9  "resultsPerPage": 10,
10  "saveHtml": false,
11  "saveHtmlToKeyValueStore": false,
12  "maxConcurrency": 10
13}

Scrape Google Search results by URL

To input URLs instead, simply replace queries with full URLs:

"queries": "<https://www.google.com/search?q=hotels+in+Seattle> \\n <https://www.google.com/search?q=hotels+in+New+York>",

⬆️ Google Search data output

The scraper stores its result in the default dataset associated with the scraper run, from which you can export it to various formats, such as JSON, XML, CSV, or Excel.

Output example (by search term)

For each Google Search results page, the dataset will contain a single record, which looks as follows. Note that the output preview will be organized in table and tabs for viewing convenience:

And here’s the equivalent of the same scraped data but in JSON. Bear in mind that some fields have example values:

1{
2    "searchQuery": {
3        "term": "hotels in seattle",
4        "url": "<http://www.google.com/search?num=100&q=hotels%20in%20seattle>",
5        "device": "DESKTOP",
6        "page": 1,
7        "type": "SEARCH",
8        "domain": "google.com",
9        "countryCode": "US",
10        "languageCode": null,
11        "locationUule": null,
12        "resultsPerPage": "100"
13    },
14    "url": "<http://www.google.com/search?num=100&q=hotels%20in%20seattle>",
15    "hasNextPage": true,
16    "serpProviderCode": "N",
17    "resultsTotal": 54000000,
18    "relatedQueries": [
19        {
20            "title": "Feedback",
21            "url": "<https://www.google.com/#>"
22        },
23        {
24            "title": "Hyatt Regency Seattle",
25            "url": "<https://www.google.com/search?num=100&q=Hyatt+Regency+Seattle&stick=H4sIAAAAAAAAAOOQUeLSz9U3MDHOSTEwNRJNzsnMTSxJVSjISU1JT1XIyC9JzSmOEoTQCpl5CsWpiSUlOamnGJG0QTlGVWbpBkVQTm5ZXnFG8ilGHv10fcPKXAujksKUNKhceVp6RVI5lGOcYZicnnSKkRuksCSl2CwlJ_cXo6gz1CEByA5pYGFcxCrqUQl0gkJQanpqXnKlQjDEQbfYJBlKnl2b9VUhNGan8tzMrdvyU0SSttVPXDDVAAAvau1f5QAAAA&sa=X&ved=2ahUKEwiRxJ_e2J__AhUVZTABHRosAFcQs9oBKAB6BAgXEAI>"
26        }
27        // ... and many more
28    ],
29    "paidResults": [
30        {
31            "title": "Hotels in Seattle, WA - Lowest Price Guarantee.",
32            "url": "<https://www.google.com/aclk?sa=l&ai=DChcSEwiMzqXe2J__AhW_gVoFHbh6BGYYABABGgJ2dQ&sig=AOD64_3PbWI_s3u1fE0Dh4Dobi56jDDCfw&q&adurl>",
33            "displayedUrl": "",
34            "description": "",
35            "emphasizedKeywords": [],
36            "siteLinks": [
37                {
38                    "title": "",
39                    "url": "<https://www.google.com/aclk?sa=l&ai=DChcSEwiMzqXe2J__AhW_gVoFHbh6BGYYABAHGgJ2dQ&sig=AOD64_3x8pu1mywduFwZgi9cwut7L9Nrvg&q=&ved=2ahUKEwiRxJ_e2J__AhUVZTABHRosAFcQh78CegQIDhAB&adurl=>",
40                    "description": ""
41                },
42                {
43                    "title": "Hotels at Great Prices",
44                    "url": "<https://www.google.com/aclk?sa=l&ai=DChcSEwiMzqXe2J__AhW_gVoFHbh6BGYYABADGgJ2dQ&sig=AOD64_1Bl0Ju75dgDPwLJAb53UYrV1xyMQ&q&adurl>",
45                    "description": ""
46                }
47                // ... and many more
48            ],
49            "type": "paid",
50            "adPosition": 1
51        },
52        {
53            "title": "Cheap Hotels in Seattle - Best Deals in Seattle",
54            "url": "<https://www.google.com/aclk?sa=l&ai=DChcSEwiMzqXe2J__AhW_gVoFHbh6BGYYABAAGgJ2dQ&sig=AOD64_27qRewPYeehK82DHa3ck7RlyilRg&q&adurl>",
55            "displayedUrl": "",
56            "description": "",
57            "emphasizedKeywords": [],
58            "siteLinks": [
59                {
60                    "title": "",
61                    "url": "<https://www.google.com/aclk?sa=l&ai=DChcSEwiMzqXe2J__AhW_gVoFHbh6BGYYABACGgJ2dQ&sig=AOD64_0sufwpcy9IZTR-ZX82HaFX1Xqc-w&q=&ved=2ahUKEwiRxJ_e2J__AhUVZTABHRosAFcQh78CegQIBxAB&adurl=>",
62                    "description": ""
63                }
64            ],
65            "type": "paid",
66            "adPosition": 2
67        }
68    ],
69    "paidProducts": [],
70    "organicResults": [
71        {
72            "title": "The Best Hotels in Seattle, WA",
73            "url": "<https://www.hotels.com/de1481165/hotels-seattle-washington/>",
74            "displayedUrl": "<https://www.hotels.com> › ... › Hotels in Washington",
75            "description": "The Inn at Virginia Mason · The Belltown Inn · Hyatt Regency Seattle · Mediterranean Inn · Hotel Max · Crowne Plaza Seattle - Downtown, an IHG Hotel · Mayflower Park ...",
76            "emphasizedKeywords": [
77                "Seattle",
78                "Hotel",
79                "Seattle",
80                "Hotel"
81            ],
82            "siteLinks": [],
83            "productInfo": {},
84            "type": "organic",
85            "position": 1
86        },
87        {
88            "title": "THE 10 BEST Hotels in Seattle, WA 2023 (from $87)",
89            "url": "<https://www.tripadvisor.com/Hotels-g60878-Seattle_Washington-Hotels.html>",
90            "displayedUrl": "<https://www.tripadvisor.com> › ... › Seattle Hotels",
91            "description": "Popular hotels in Seattle right now · 1. Embassy Suites by Hilton Seattle Downtown Pioneer Square · 1,293 reviews · 2. Hyatt Regency Seattle · 1,917 reviews · 3.",
92            "emphasizedKeywords": [
93                "hotels in Seattle"
94            ],
95            "siteLinks": [],
96            "productInfo": {},
97            "type": "organic",
98            "position": 2
99        }
100        // ... and many more
101    ],
102    "peopleAlsoAsk": [
103        {
104            "question": "Is it better to stay in downtown Seattle or?",
105            "answer": "For most visitors, the best area to stay is downtown Seattle (and near Pike Place Market). Downtown is where most of Seattle's best hotels, top restaurants, attractions, and shopping are located – all within an area dense, compact, and walkable.",
106            "url": "<https://santorinidave.com/best-places-seattle#:~:text=For%20most%20visitors%2C%20the%20best,dense%2C%20compact%2C%20and%20walkable.">,
107            "title": "WHERE TO STAY in SEATTLE - Best Areas & Neighborhoods",
108            "date": "May 9, 2023"
109        },
110        {
111            "question": "Is it easy to get around Seattle without a car?",
112            "answer": "You can easily get around Seattle without a car. Places like Pike Place Market, the Seattle Center, the waterfront, and a lot of neighborhoods are walkable.",
113            "url": "<https://travellemming.com/getting-around-seattle/#:~:text=You%20can%20easily%20get%20around,lot%20of%20neighborhoods%20are%20walkable.">,
114            "title": "Getting Around Seattle (A Local's Transportation Guide)",
115            "date": "Jan 13, 2023"
116        },
117        {
118            "question": "Where not to stay in downtown Seattle?",
119            "answer": "Avoid downtown Seattle's 3rd Avenue, particularly between Pike and Pine, and James and Yesler. Pioneer Square, SoDo, the International District, First Hill, and certain parts of Belltown can be a bit sketchy. That said, visitors should exercise caution when exploring at night.",
120            "url": "<https://travellemming.com/is-seattle-safe/#:~:text=Avoid%20downtown%20Seattle's%203rd%20Avenue,caution%20when%20exploring%20at%20night.">,
121            "title": "Is Seattle Safe in 2023? (Honest Info From a Local) - Travel Lemming",
122            "date": "Feb 16, 2023"
123        },
124        {
125            "question": "Should I stay in West Seattle?",
126            "answer": "West Seattle Don't get me wrong: West Seattle is a great place. In fact, I highly recommend taking a half-day or day to visit the area if you have the time during your Seattle trip. Especially along Alki Beach, you can enjoy sweeping views of Puget Sound and the Seattle skyline, plus enjoy local food and drink.",
127            "url": "<https://www.valisemag.com/where-to-stay-seattle/#:~:text=5.-,West%20Seattle,enjoy%20local%20food%20and%20drink.">,
128            "title": "The 10 Best Areas to Stay in Seattle (& Where Not To!) - Valerie & Valise",
129            "date": "Sep 29, 2020"
130        }
131    ],
132    "customData": {
133        "pageTitle": "hotels in seattle - Google Search"
134    }
135}

You can download the results directly from the platform using a button or from the Get dataset items API endpoint:

<https://api.apify.com/v2/datasets/[DATASET_ID]/items?format=[FORMAT]>

where [DATASET_ID] is the ID of the dataset and [FORMAT]can be csv, html, xlsx, xml, rss or json.

🧑‍🏫 How do I use Google SERP Scraper?

To understand how to set up and run the Google SERP Scraper (or Google SERP API), check out our step-by-step guide on how to scrape Google Search with screenshots and examples or watch a short video tutorial ▷ on YouTube.

❓FAQ

📚 Resources on how to use Google SERP API to scrape Google

1. Is web scraping legal? - blogpost on ethical scraping.
2. Platform pricing page with pricing specifications.
3. Video guide ▷ on how to count the usage of Google Search API.
4. Step-by-step guide on how to use Google SERP scraper.
5. Video tutorial ▷ on how to use Google SERP scraper.
6. Input tab with all the technical parameters of this scraper.
7. A list of other Google-related scrapers.

How do I scrape millions of results from Google?

Please note that, although Google always shows that it has found millions of results, Google will never display more than a few hundred results for a single search query 🤥 If your goal is to get as many results as possible, try creating many similar queries and combine different parameters and locations.

How do I scrape Google ads?

Despite the fact that scraping Google is the best shot we've got at seeing objective Google results, displayed paid results are still heavily dependent on your location and browsing history. It’s also up to Google to choose which ads to show to which user (hello, cookies 🍪). It might be underwhelming to see less results than you'd expected. Most probably, Google's precise algorithm is the reason for it.

Is it legal to scrape Google search results?

Web scraping is legal if you are extracting publicly available data, but you should respect boundaries such as personal data and intellectual property regulations. Rule of thumb is: you should only scrape personal data if you have a legitimate reason to do so, factoring in Google's Terms of Use as well. If you're unsure whether your reason is legitimate, consult your lawyers. We also recommend that you read our blog post on the subject: is web scraping legal?

What are other tools for scraping Google services?

👁 Google Lens Scraper	📍 Google Maps Scraper
▶️ Google Play Reviews Scraper	🛍 Google Shopping Scraper
📈 Google Trends Scraper	⭐️ Google Maps Reviews Scraper
📩📍 Google Maps Email Extractor	🤟 Google Datasets Translator

How can I use data scraped from Google Search?

Google SERP API has a lot to offer in terms of how extracted Google data can be applied:

🔸 Use it for search engine optimization (SEO) and monitor how your website performs on Google for certain queries over time.

🔹 Monitor how frequently a search term has been used on Google, and how it compares with total search volume.

🔸 Analyze display ads for a given set of keywords.

🔹 Monitor your competition in both organic and paid results.

🔸 Build a URL list for certain keywords. This is useful if, for example, you need good relevant starting points when scraping web pages containing specific phrases.

🔹 Analyze Google algorithm and identify its main trends

Can I integrate Google Scraper with other apps?

Last but not least, this Google SERP API can be connected with almost any cloud service or web app thanks to integrations on the Apify platform. You can integrate with Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. Or you can use webhooks to carry out an action whenever an event occurs, e.g. get a notification whenever Google Search Results Scraper successfully finishes a run.

Can I use Google Scraper with the API?

The Apify API gives you programmatic access to the Apify platform. The API is organized around RESTful HTTP endpoints that enable you to manage, schedule, and run Apify Actors. The API also lets you access any datasets, monitor actor performance, fetch results, create and update versions, and more.

To access the API using Node.js, use the apify-client NPM package. To access the API using Python, use the apify-client PyPi package.

Check out the Apify API reference docs for full details or click on the API tab for code examples. You can also follow this video guide.

How to get one search result per row

Simply choose the Export view for Organic results and/or Paid results, it automatically spreads each result into a separate row. For API access, you can add &view=paid_results or &view=organic_results to the URL and with the API client, you can do the same using the view field.

An organic result is represented using the following format:

1{
2  "searchQuery": {
3    "term": "laptop",
4    "device": "DESKTOP",
5    "page": 1,
6    "type": "SEARCH",
7    "domain": "google.com",
8    "countryCode": "US",
9    "languageCode": "en",
10    "locationUule": null,
11    "resultsPerPage": 10
12  },
13  "type": "organic",
14  "position": 1,
15  "title": "Laptops & Notebook Computers - Best Buy",
16  "url": "<https://www.bestbuy.com/site/computers-pcs/laptop-computers/abcat0502000.c?id=abcat0502000>",
17  "displayedUrl": "<https://www.bestbuy.com> › Computers & Tablets",
18  "description": "Shop Best Buy for laptops. Work & play from anywhere with a notebook computer. We can help you find the best laptop for your specific needs in store and online.",
19  "emphasizedKeywords": "laptops | laptop",
20  "productInfo": {}
21}

A paid result has an adPosition field instead of position and "type": "paid". Paid result position is calculated separately from the organic results.

When using a tabular format such as csv or xls, you'll get a table where each row contains just one organic result. For more details about exporting and formatting the dataset records, please see the documentation for the Get dataset items API endpoint.

Not your cup of tea? Build your own scraper.

This Google SERP API doesn’t exactly do what you need? You can always build your own! We have various scraper templates in Python, JavaScript, and TypeScript to get you started. Alternatively, you can write it from scratch using our open-source library Crawlee. You can keep the scraper to yourself or make it public by adding it to Apify Store (and find users for it).

Or let us know if you need a custom scraping solution.

Your feedback

If you are not sure that the results are complete and of good quality, each run stores the full HTML page to the default Key-Value Store. You can view the KVS by clicking on it and comparing the results.

We’re always working on improving the performance of our Actors and monitoring the quality but we are happy for any reports. So if you’ve got any technical feedback for this Google SERP API or simply found a bug, please create an issue on the Actor’s Issues tab in Apify Console.