Google Search Scraper - SERP Results & Rich Snippets

Extract structured data from Google search results with precision. This actor leverages Apify's Google SERP proxy to scrape real Google search engine results pages (SERPs) and return organic results, featured snippets, ads, and "People Also Ask" questions in clean JSON format.

Features

• Organic Search Results: Title, URL, description, position, domain
• Featured Snippets: Automatically detects and extracts knowledge panels and featured snippets
• People Also Ask: Captures related questions and answers
• Multi-Country Support: US, UK, Canada, Australia, Germany, France, Japan, Brazil, India, Mexico
• Language Support: Search results in multiple languages
• Bulk Operations: Process multiple queries in a single run
• Structured Output: Clean JSON data ready for analysis or integration

Use Cases

• SEO Research: Analyze competitor rankings and SERP features
• Market Intelligence: Track keyword rankings and featured snippet opportunities
• Competitive Analysis: Monitor search results for market trends
• Content Strategy: Identify question-based content opportunities from "People Also Ask"
• Lead Generation: Extract business websites from search results
• Academic Research: Gather search trend data at scale

Input

The actor accepts an input object with the following structure:

{
  "queries": ["python programming", "web scraping"],
  "maxResults": 10,
  "country": "US",
  "language": "en"
}

Input Parameters

• queries (required): Array of search queries (e.g., ["machine learning", "AI trends"])
• maxResults: Number of results per query (1-100, default: 10)
• country: Search country (US, UK, CA, AU, DE, FR, JP, BR, IN, MX) - default: US
• language: Result language code (en, de, fr, es, ja, pt) - default: en

Output

Results are pushed to the default dataset as individual JSON records:

{
  "query": "python programming",
  "position": 1,
  "type": "organic_result",
  "title": "Welcome to Python.org",
  "url": "https://www.python.org",
  "domain": "python.org",
  "description": "The official home of the Python Programming Language."
}

Result Types

• organic_result: Standard search result (title, URL, description, position)
• featured_snippet: Featured snippet from knowledge panel
• people_also_ask: Related questions from "People Also Ask" section
• ad: Sponsored search result (if detected)
• error: Error record if query fails

Pricing

This actor uses Pay-Per-Event (PPE) pricing:

• $0.005 per actor start
• $0.003 per result extracted

For 10 queries with 10 results each: $0.005 + (100 × $0.003) = $0.305

Technology Stack

• Python 3.12 with asyncio
• BeautifulSoup4: HTML parsing
• httpx: Async HTTP client
• Apify SDK: Job scheduling and data export
• Apify Google SERP Proxy: Real Google HTML via proxy network

Rate Limiting

Google enforces rate limits on search requests. The actor handles rate limiting gracefully:

• Automatic retry on 429 (Too Many Requests)
• Exponential backoff
• Proxy rotation via Apify's residential proxy network

Limitations

• Google actively blocks automated scraping; this actor uses Apify's official proxy to mitigate blocks
• Search results vary by location, IP, and query history
• Some specialized results (maps, images, videos) may not be captured in the organic results list
• Requires active Apify account with proxy access

Examples

1. Basic SEO Research

{
  "queries": ["best coffee shops near me", "coffee trends 2026"],
  "maxResults": 20,
  "country": "US"
}

2. Multi-Language Analysis

{
  "queries": ["machine learning"],
  "maxResults": 15,
  "country": "DE",
  "language": "de"
}

3. Competitive Intelligence

{
  "queries": ["our company name", "competitor name", "industry keyword"],
  "maxResults": 30
}

Support

For issues or feature requests, please contact support or check the Apify marketplace documentation.

License

Proprietary - NexGenData