2 hours trial then $5.55/month - No credit card required now

SERP

yuriy_chistyakov/serp

2 hours trial then $5.55/month - No credit card required now

Images are considered as a valuable field of Google Search Engine Results Page (SERP) and always added to Dataset if any.

How it works

This code is a JavaScript script that uses Cheerio to scrape data from Google Search Engine Results Pages.

The crawler starts with startUrl https://www.google.com/search?q=${q}&start=0&${num}&lr=lang_en&hl=en where "q" and "num" parameters provided from the input q and num fields defined by the input schema. Any spaces in the q field being encoded to plus(+) automatically.
The crawler uses requestHandler for each URL to extract the data from the page with the Cheerio library and, while the results array is not empty, saves the url, pageTitle and results of each page to the Dataset, otherwise Actor exits. Every result in array consists of site, link, title, description, image. After the data has been saved, the crawler enqueues request to the next page with the "start" parameter increased by the value of "num".
Number of scraped pages is limited by Max Requests per Crawl field from the input schema. It also logs out the url of each page visited.

Included features

Serpifier - a free tool I created for visualizing datasets produced by SERP

Let's take a closer look at how it works using serpifier

By entering a search query and leaving the num and maxRequestsPerCrawl parameters unchanged, we get the following graph:

num: 100, maxRequestsPerCrawl: 100

Now let's limit the number of results by setting num to 10 and maxRequestsPerCrawl to 10. That is, 10 pages with a maximum of 10 results per page:

num: 10, maxRequestsPerCrawl: 10

And here is the result with the maxRequestsPerCrawl set to 20. The graph will look something like this:

num: 10, maxRequestsPerCrawl: 20 In fact, we don’t know the number of search results in advance, so we’re just playing with the parameters.

Curious, how looks the graph with num: 2 and maxRequestsPerCrawl: 100?

num: 2, maxRequestsPerCrawl: 100

By setting a sitesearch input parameter to "apollographql.com", we will get a graph similar to the one shown below:

sitesearch: "apollographql.com"

Developer

Yuriy Chistyakov

Actor metrics

1 monthly user
0 stars
100.0% runs succeeded
Created in Jul 2024
Modified 2 months ago

Categories

SEO tools

Developer tools

Integrations

FREE Google SERP First Page

adrian_horning/free-google-serp-first-page

Scrape the first page of Google for FREE! Returns the links and the title of the links.

Adrian | The Web Scraping Guy

247

Google SERP Scraper

quaking_pail/google-serp-scraper

Extracting detailed data from Google Search Engine Results Pages (SERPs) specific to search, including organic and paid listings, descriptions, URLs, and emphasized keywords. Ideal for digital marketers, SEO professionals, and market researchers. It will not scrap each site (easy and light run).

AI_Builder

Google Condensed SERP Scraper

quaking_pail/google-condensed-serp-scraper

It condenses search engine results into a single, well-formatted text summary. Ideal for feeding into a language model (LLM), making it perfect for lead generation, research, or quickly finding specific details from searches. This process is quite resource intensive hence a renting model.

AI_Builder

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Apify

47.3k

169

Google Search Videos Scraper

web.harvester/google-videos-scraper

This Google Search Videos Scraper enables you to scrape Google Search Engine Results Pages (SERPs) for videos result. Select a country or language, extract custom attributes, and download your data; no coding is needed.

Web Harvester

105

Keyword Finder

ahmed_elkurdi/keyword-finder

Keyword Finder : Keyword Research Scrapper, Get search volumes Keyword Ranking Difficulty, Long Tail Keyword Suggestion and SERP metrics from popular SEO Service Websites .

Ahmed Elkurdi

Fast Google Search Results Scraper

serping/fast-google-search-results-scraper

Scrape Google Search Engine Results Page (SERP). Select the country or language and extract organic and paid results, ads, queries, featured snippets, People Also Ask, Top Stories, inline videos, reviews, like a Google SERP API.

SERP.ing

Keyword Difficulty API ✅ Free

curious_coder/keyword-difficulty-api

Find keyword difficulty ranks in bulk from ahrefs, get websites ranking in SERP

Curious Coder

Google Search Scraper

eunit/google-search-scraper

Uncover valuable insights! This Google Search Scraper extracts organic & paid results, ads, queries, "People Also Ask," prices, and reviews – like a SERP API: export data (JSON, CSV, Excel) & schedule scrapes.