Google Shopping Price Comparison Scraper

An Apify Actor that scrapes product listings, prices, merchants, shipping information, and ratings from Google Shopping. Designed for competitive pricing intelligence, market research, and e-commerce analysis.

Features

• Multiple search queries — run many product searches in a single execution
• Structured product data — title, price, currency, merchant, rating, reviews, shipping, condition, image
• JSON-LD + HTML parsing — extracts structured data when available, falls back to HTML card parsing with multiple selector strategies
• Price filtering — optional min/max price filters per query
• Country targeting — set geolocation via country code (US, GB, DE, FR, JP, etc.)
• Rate limiting — configurable delays between requests to avoid blocking
• Graceful abort — cleanly shuts down when the Actor is stopped mid-execution
• Chrome user-agent rotation — rotates user-agent strings across queries
• Deduplication — merges results from JSON-LD and HTML sources by URL

Input

Field	Type	Default	Description
searchQueries	string[]	["iPhone 15 case"]	One or more product search terms
maxResults	integer	50	Max product results per query (1–200)
country	string	"US"	Two-letter country code (US, GB, DE, FR, etc.)
minPrice	number	null	Optional minimum price filter
maxPrice	number	null	Optional maximum price filter

Example input

{
  "searchQueries": ["iPhone 15 case", "wireless headphones", "USB-C hub"],
  "maxResults": 30,
  "country": "US",
  "minPrice": 5,
  "maxPrice": 100
}

Output

Each dataset item contains:

Field	Type	Description
title	string	Product name
url	string	Google Shopping product URL
merchant	string	Store/seller name
price	number	Current price
currency	string	ISO 4217 currency (USD, EUR, GBP...)
originalPrice	number?	Strikethrough original price (if on sale)
shippingCost	number?	Shipping cost (0 = free)
shippingInfo	string?	Raw shipping description
rating	number?	Average rating (0–5)
reviewCount	integer?	Number of reviews
condition	string?	new, used, refurbished, or null
imageUrl	string?	Product thumbnail URL
taxInfo	string?	Tax-related details
searchQuery	string	The search term that produced this result
scrapedAt	string	ISO 8601 scrape timestamp
source	string	"jsonld" or "html" (data origin)

Usage

Local development

# Install dependencies
pip install -r requirements.txt

# Run locally with Apify CLI
cd /root/workspace/google-shopping-scraper
apify run --purge

Deploy to Apify

$apify push

Or import the Git repository into the Apify Console.

How it works

1. URL construction — builds search URLs with tbm=shop (Shopping tab), hl=en, and gl (geolocation) parameters
2. HTTP request — uses httpx.AsyncClient with Chrome user-agent and retry logic
3. JSON-LD parsing — extracts structured product data from <script type="application/ld+json"> blocks
4. HTML card parsing — falls back to parsing product card divs with multiple CSS selector strategies
5. Deduplication — merges results from both sources, preferring JSON-LD data
6. Batched push — pushes results to the Apify dataset in batches of 25

Limitations

• Google adapts HTML class names over time — the fallback selector system handles this but may occasionally miss results
• Pagination support is limited (Google loads most results dynamically via JavaScript)
• Some merchant sites block or redirect Google Shopping referral links
• Rate limits may apply for high-volume scraping

License

MIT