Google SERP All-in-One Scraper — AI Overviews, PAA, Ads & Knowledge Panel

One actor. One query. Every Google SERP feature in one structured record. MCP-ready for AI agents.

What it does

This actor takes a list of Google search queries and returns ONE dataset record per query containing every SERP feature block Google shows for that query:

• Organic results — title, URL, domain, snippet, displayed URL, sitelinks, rich snippet
• Sponsored ads — top + bottom blocks, advertiser, sitelinks, extensions
• AI Overviews (AIO) — full answer text, paragraph blocks, cited sources with positions
• People Also Ask (PAA) — questions, snippets, source links
• Featured Snippet — paragraph / list / table / video with source
• Knowledge Panel — entity card with title, type, description, image, website, social profiles
• Local Pack — Local 3-pack businesses with rating, reviews, address, category
• Related searches — Google's suggested follow-up queries
• Video carousel — top videos with source, duration, thumbnail
• Image pack — image carousel with thumbnails
• Shopping results — sponsored product listings with price, seller, rating
• Top Stories — news carousel with publisher, date

Multi-country (40+), multi-language (19), desktop or mobile SERP layout.

When to use

Use this actor when you need structured Google SERP data with EVERY feature — not just blue links. Built for:

• SEO agencies tracking AI Overview citations — see which domains Google quotes for each query
• Brand visibility monitoring — detect when your brand appears in AIO, Featured Snippet, Knowledge Panel, or Local Pack
• Rank tracking with full SERP context — organic rank + ad density + SERP feature mix
• Competitive intelligence — see competitor ads, ad copy, ad extensions for any query
• AI-search research — feed AIO citations into RAG/LLM pipelines
• AI agents (Claude, GPT, Gemini) — one MCP tool call returns structured JSON ready for reasoning

When NOT to use

• For raw Google Search URL scraping → use khadinakbar/scrape-google-serp
• For AI Overviews only → use khadinakbar/google-ai-overviews-scraper (cheaper, AIO-focused)
• For LIVE ads with sitelinks-only → use khadinakbar/google-ads-scraper
• For Google News only → use khadinakbar/google-news-scraper
• For Google Shopping product feeds → use khadinakbar/google-shopping-scraper
• For brand monitoring across multiple AI engines (Perplexity, ChatGPT, Gemini) → use khadinakbar/ai-search-brand-monitor

Output

One record per query. Sample shape (truncated for brevity):

{
  "query": "best ai search tools 2026",
  "countryCode": "us",
  "languageCode": "en",
  "device": "desktop",
  "source": "serpapi-managed",
  "serpUrl": "https://www.google.com/search?q=best+ai+search+tools+2026&hl=en&gl=us&num=10",
  "totalResultsEstimate": 18400000,
  "searchTimeSeconds": 0.42,
  "organic": [
    {
      "position": 1,
      "title": "Top 10 AI Search Engines in 2026",
      "url": "https://example.com/ai-search-2026",
      "domain": "example.com",
      "displayedUrl": "example.com › blog",
      "snippet": "A ranked list of the most-used AI search engines...",
      "sitelinks": []
    }
  ],
  "ads": [
    {
      "position": 1,
      "block": "top",
      "title": "Perplexity Pro — AI Search Engine",
      "url": "https://www.perplexity.ai/pro",
      "advertiser": "Perplexity AI"
    }
  ],
  "aiOverview": {
    "present": true,
    "text": "AI search tools combine large language models with web retrieval...",
    "textBlocks": ["paragraph 1...", "paragraph 2..."],
    "citations": [
      { "position": 1, "domain": "perplexity.ai", "url": "https://perplexity.ai/", "title": "Perplexity AI" }
    ],
    "citationCount": 6,
    "uniqueDomainCount": 5
  },
  "peopleAlsoAsk": [
    { "position": 1, "question": "Which AI search engine is best?", "answer": "Perplexity, ChatGPT Search, and Google AI Overviews are the most popular..." }
  ],
  "featuredSnippet": null,
  "knowledgePanel": null,
  "localPack": [],
  "relatedSearches": ["best free ai search engines", "perplexity vs chatgpt search", "google ai overviews"],
  "videos": [],
  "images": [],
  "shoppingResults": [],
  "topStories": [],
  "scrapedAt": "2026-06-13T12:50:00Z",
  "warnings": []
}

Pricing — Pay Per Event

Event	Price	When charged
Actor start	$0.00005	Once per run (scaled by RAM)
SERP page (managed)	$0.008	Per query returned via our managed SerpApi backend (default — full feature coverage including AI Overviews, Knowledge Panel, Local Pack)
SERP page (Camoufox fallback)	$0.003	Per query when forceCamoufox: true is set. Cheaper but degraded coverage

Typical cost for 100 queries (default managed path): ~$0.80.

Pay-per-usage (compute + proxy passthrough) is also available — pick whichever fits your job better at run time.

Backend

This actor uses a managed SerpApi backend by default. You do NOT need a SerpApi account — your Apify billing covers everything. Set forceCamoufox: true only if you want to experiment with the cheaper Camoufox fallback (currently degraded coverage; not recommended for prod).

Input example

{
  "queries": [
    "best ai search tools 2026",
    "apify google scraper",
    "remote software engineer jobs"
  ],
  "countryCode": "us",
  "languageCode": "en",
  "device": "desktop",
  "extractFeatures": ["organic", "ads", "aiOverview", "peopleAlsoAsk", "featuredSnippet", "knowledgePanel", "localPack", "relatedSearches"],
  "maxOrganicResults": 10
}

Architecture

Hybrid extraction with circuit breaker:

1. Camoufox primary — headless Firefox with anti-fingerprint masking (Apify Residential proxy, country-aligned).
2. Captcha detect → blocked HTML pages dump to KV store under DEBUG-* keys, session retires.
3. Circuit breaker — 3 consecutive captchas open the circuit; remaining queries skip Camoufox and go straight to SerpApi.
4. SerpApi fallback — managed key. Returns ALL SERP feature blocks in full fidelity.
5. Per-query upgrade — if Camoufox returns a near-empty record (no organic, no ads, no AIO), upgrade via SerpApi even when not blocked.
6. safePushData — schema-strip retry on any pushData failure.
7. Honest-fail — if every query fails AND zero items pushed, the run marks FAILED with a clear status; partial success stays SUCCEEDED.

MCP-ready

Tool name: apify--google-serp-all-in-one-scraper

Once published, this actor is callable from any MCP client (Claude Desktop, Claude Code, GPT custom GPTs, Gemini, Cursor) via mcp.apify.com. Tool description is written for agent consumption — verb + scope + return shape + price signal in <500 chars.

Country support (40 codes)

us, gb, ca, au, de, fr, es, it, nl, br, mx, ar, in, jp, kr, sg, ae, sa, za, ng, pl, se, no, dk, fi, ie, be, ch, at, pt, gr, tr, ph, id, my, th, vn, il, cl, co, pe

Language support (19 codes)

en, es, fr, de, it, pt, nl, ru, ja, ko, zh, ar, hi, tr, pl, sv, no, da, fi

Cross-sells

Pair with these portfolio actors:

• khadinakbar/ai-search-brand-monitor — multi-engine brand visibility (this + Perplexity + ChatGPT + Gemini)
• khadinakbar/google-ai-overviews-scraper — AIO-only deep dive
• khadinakbar/google-ads-transparency-scraper — historical ad creatives
• khadinakbar/google-trends-scraper — search-volume trends for the same queries
• khadinakbar/google-news-scraper — Top Stories carousel deep-dive

Limits & disclaimers

• This actor scrapes publicly-accessible Google search results only. No login, no impersonation.
• Google rotates SERP HTML quarterly — Camoufox selectors are best-effort. SerpApi fallback is the reliability backbone.
• Anti-bot blocks happen. The circuit breaker + SerpApi route exists precisely because Google blocks headless browsers consistently.
• You are responsible for compliance with Google Terms of Service, local laws, and any data-processing regulations applicable to your use case.
• Rate-limit your queries. Don't run 10,000 queries from a single account in one minute. The actor enforces internal concurrency = 2 to behave nicely.

Support

Issues + feature requests: Apify Store actor page → Issues tab.

---

Built by khadinakbar — full Apify portfolio at apify.com/khadinakbar.