2 hours trial then $5.55/month - No credit card required now

SERP

yuriy_chistyakov/serp

2 hours trial then $5.55/month - No credit card required now

Images are considered as a valuable field of Google Search Engine Results Page (SERP) and always added to Dataset if any.

How it works

This code is a JavaScript script that uses Cheerio to scrape data from Google Search Engine Results Pages.

The crawler starts with startUrl https://www.google.com/search?q=${q}&start=0&${num}&lr=lang_en&hl=en where "q" and "num" parameters provided from the input q and num fields defined by the input schema. Any spaces in the q field being encoded to plus(+) automatically.
The crawler uses requestHandler for each URL to extract the data from the page with the Cheerio library and, while the results array is not empty, saves the url, pageTitle and results of each page to the Dataset, otherwise Actor exits. Every result in array consists of site, link, title, description, image. After the data has been saved, the crawler enqueues request to the next page with the "start" parameter increased by the value of "num".
Number of scraped pages is limited by Max Requests per Crawl field from the input schema. It also logs out the url of each page visited.

Included features

Serpifier - a free tool I created for visualizing datasets produced by SERP

Let's take a closer look at how it works using serpifier

By entering a search query and leaving the num and maxRequestsPerCrawl parameters unchanged, we get the following graph:

num: 100, maxRequestsPerCrawl: 100

Now let's limit the number of results by setting num to 10 and maxRequestsPerCrawl to 10. That is, 10 pages with a maximum of 10 results per page:

num: 10, maxRequestsPerCrawl: 10

And here is the result with the maxRequestsPerCrawl set to 20. The graph will look something like this:

num: 10, maxRequestsPerCrawl: 20 In fact, we don’t know the number of search results in advance, so we’re just playing with the parameters.

Curious, how looks the graph with num: 2 and maxRequestsPerCrawl: 100?

num: 2, maxRequestsPerCrawl: 100

By setting a sitesearch input parameter to "apollographql.com", we will get a graph similar to the one shown below:

sitesearch: "apollographql.com"

Developer

Yuriy Chistyakov

Actor Metrics

1 monthly user
0 No stars yet
>99% runs succeeded
Created in Jul 2024
Modified 5 months ago

Categories

SEO tools

Developer tools

Integrations

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

50.9k

258

FREE Google SERP First Page

adrian_horning/free-google-serp-first-page

Scrape the first page of Google for FREE! Returns the links and the title of the links.

Adrian | The Web Scraping Guy

345

Google SERP Scraper

quaking_pail/google-serp-scraper

Extracting detailed data from Google Search Engine Results Pages (SERPs) specific to search, including organic and paid listings, descriptions, URLs, and emphasized keywords. Ideal for digital marketers, SEO professionals, and market researchers. It will not scrap each site (easy and light run).

AI_Builder

Google Condensed SERP Scraper

quaking_pail/google-condensed-serp-scraper

It condenses search engine results into a single, well-formatted text summary. Ideal for feeding into a language model (LLM), making it perfect for lead generation, research, or quickly finding specific details from searches. This process is quite resource intensive hence a renting model.

AI_Builder

DuckDuckGo Serp API

canadesk/duckduckgo-serp-api

Search for text, news, images, videos, maps or start an AI chat with DuckDuckGo! Alternative to Google SerpApi. Fast and cheap.

Canadesk Support

SERP - Google Search Results Crawler API

rajtripathi2580/jscrawler

SERP - Google Search Results Crawler API 1. Fetch unlimited search results from Google. 2. Retrieve titles, snippets, and links for each result. 3. Get real-time data for accurate insights. 4. Customize queries by keywords, location, and language.

Raj

Google Search Videos Scraper

web.harvester/google-videos-scraper

This Google Search Videos Scraper enables you to scrape Google Search Engine Results Pages (SERPs) for videos result. Select a country or language, extract custom attributes, and download your data; no coding is needed.

Web Harvester

122

Fast Google Search Results Scraper

serping/fast-google-search-results-scraper

Scrape Google Search Engine Results Page (SERP). Select the country or language and extract organic and paid results, ads, queries, featured snippets, People Also Ask, Top Stories, inline videos, reviews, like a Google SERP API.

SERP.ing

133

Keyword Difficulty API

curious_coder/keyword-difficulty-api

Find keyword difficulty ranks in bulk from ahrefs, get websites ranking in SERP

Curious Coder

PixelSERP

pixelserp/pixelserp

Get instant access to comprehensive Google SERP data with our high-performance API. Extract organic results, ads, featured snippets, and more from 195+ countries with 99.9% uptime. Perfect for SEO tools, rank tracking, and market research.