Google Search Results Scraper avatar
Google Search Results Scraper

Pricing

$3.50 / 1,000 results

Go to Store
Google Search Results Scraper

Google Search Results Scraper

apify/google-search-scraper

Developed by

Apify

Maintained by Apify

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

4.7 (22)

Pricing

$3.50 / 1,000 results

434

Monthly users

2.5k

Runs succeeded

>99%

Response time

5.8 days

Last modified

5 hours ago

XR

AI Overviews are sometimes truncated after the first paragraph and/or before the list content

Open
xlsa-rbm opened this issue
3 days ago

google-search-scraper does return AI overview data, but oftentimes it is truncating the data after the first paragraph or two. AIOs often have a lead in paragraph and then some text that ends in a colon, like this:

A brief summary of the search results for the query.

Here's some more information about topic of query:

Multi-section list goes here

I'm observing that the scraper often truncates the AIO either after the first paragraph or after the second lead-in paragraph, omitting the list content.

Observations: AIO content is basically a bunch of child divs inside a container div. Each of the first 2 paras is in a separate div, then each heading/subheading and list section is also in a div.

At least in some cases Google injects a <style> tag after the first two paragraph-containing divs (so right after the "Here's some more information about topic of query:" section.) Perhaps something in the markup extraction is breaking on this? Unfortunately, the HTML the actor saves to KV doesn't contain the rendered AIO output, so I can't verify that it is there in the DOM/markup the actor sees.

Pricing

Pricing model

Pay per result 

This Actor is paid per result. You are not charged for the Apify platform usage, but only a fixed price for each dataset of 1,000 items in the Actor outputs.

Price per 1,000 items

$3.50