Google SERP + AI Overviews API

SerpAPI-style Google search JSON — AI Overview answer + citations + organic results + knowledge graph + top stories + local results in one response per query, at a fraction of the cost.

• Batch up to 1000 queries per run (array input, one API call) — processed with 10-way concurrency (configurable 1–30), rows stream to the dataset as they complete (a timed-out run still delivers partial results)
• AI Overview parsed into structured text_blocks (paragraphs, headings, lists) + references (citation title + link + snippet) — including deferred AIOs Google generates asynchronously (resolved via Google's own /async/folsrch endpoint, no browser rendering, no third-party API)
• Rich SERP surface: organic results, knowledge graph (facts), top stories, local results (rating/reviews), People Also Ask, discussions & forums
• Geo targeting via location (e.g. "New York, NY", "Kuala Lumpur") — Google uule encoding, same as SerpAPI
• Country (gl) + language (hl) targeting
• Pay per search — $0.005 per completed query (badge $5.00 / 1,000 searches). Queries that return no parseable content (status Empty) or error out are not charged.
• Buyer pays platform usage on top ($0.004/query) — **$0.009 all-in per query**, ~$1.05 for 120 queries

Input

{
  "queries": ["best crm for startups", "what is retrieval augmented generation"],
  "gl": "us",
  "hl": "en",
  "maxResults": 10,
  "location": "New York, NY",
  "retryDeferredAIO": true,
  "retryDeferredMaxFetches": 2,
  "captureRawHtml": false,
  "concurrency": 10,
  "retryDeferredPoll": false
}

Field	Type	Default	Description
queries	array of strings	required	Search queries (max 1000 per run)
gl	string	us	Two-letter country code (us, my, uk, sg…)
hl	string	en	Two-letter language code (en, ms, zh…)
maxResults	integer	10	Max organic results per query (1–10)
location	string	(proxy IP geo)	Geo target, e.g. "New York, NY". Google uule-encoded server-side.
retryDeferredAIO	boolean	true	Resolve deferred AI Overviews (Google generates some answers asynchronously). Uses the embedded /async/folsrch template via a datacenter proxy. Extra cost only on affected queries. false = always single fetch.
retryDeferredMaxFetches	integer	2	Max total fetches per query when a deferred AI Overview shell is detected (1–5). 1 = single fetch, same as retryDeferredAIO: false.
captureRawHtml	boolean	false	Save the raw SERP HTML for every query to the key-value store (key raw-<query>-<ts>.html). For building golden test corpora / debugging selector drift. Adds ~200–400KB per query to KV storage.
concurrency	integer	10	Parallel queries (1–30). A/B measured: c5=34s, c10=35s, c20=65s for 20 queries — the SERP proxy pool is the bottleneck, so beyond ~10 workers latency balloons. 10 = same throughput as 5 with headroom for AIO-heavy sets.
retryDeferredPoll	boolean	false	Legacy poll re-fetch for deferred AIOs (re-fetch up to retryDeferredMaxFetches times, keep richest). Off by default — folsrch resolves 90%+ of deferred AIOs; the poll recovered 0/120 in the stress test. Enable only if folsrch regresses.

Output

One dataset item per query:

{
  "ai_overview": {
    "text_blocks": [
      { "type": "paragraph", "snippet": "The best CRM for a startup depends on your core workflow…" },
      { "type": "list", "list": [{ "title": "…" }] }
    ],
    "references": [
      { "title": "CRM for Startups: Free Software to Scale Your Business", "link": "https://www.hubspot.com/products/crm/startups", "snippet": "…", "index": 0 }
    ],
    "resolved_by": "folsrch"
  },
  "organic_results": [
    { "position": 1, "title": "…", "url": "https://…", "displayed_link": "…", "snippet": "…" }
  ],
  "people_also_ask": ["…", "…"],
  "search_metadata": { "status": "Success", "created_at": "…", "total_time_taken_ms": 3088, "google_url": "…" },
  "search_parameters": { "q": "best crm for startups", "gl": "us", "hl": "en", "location": "New York, NY" },
  "knowledge_graph": {
    "title": "…",
    "description": "…",
    "source": { "name": "Wikipedia", "link": "…" },
    "facts": [{ "label": "Born", "value": "…" }]
  },
  "top_stories": [ { "title": "…", "url": "…", "source": "The Economist" } ],
  "local_results": [ { "title": "…", "rating": "4.9", "reviews": 534 } ],
  "discussions_and_forums": [ { "title": "…", "url": "…", "source": "reddit" } ],
  "selector_strategy": {
    "organic_results": "primary",
    "people_also_ask": "primary",
    "knowledge_graph": "fallback",
    "top_stories": "structural",
    "local_results": "primary",
    "discussions_and_forums": "primary"
  }
}

• ai_overview.resolved_by is folsrch (deferred answer resolved via Google's async endpoint), poll (recovered via re-fetch), or absent (answer was inline in the SERP).
• Modules are null when Google returns none for that query.
• search_metadata.status is Success (content parsed), Empty (no parseable content — not charged), or Error (fetch failed after retries — not charged).
• search_metadata.drift_evidence (present only on Empty rows) is the key of the raw HTML sample saved to the actor's key-value store — grab it for instant diagnosis of selector drift.
• selector_strategy records which parser strategy fired per module: primary (current Google classes), fallback (stable anchors), or structural (class-free heuristics). primary everywhere = healthy; fallback/structural = Google rotated classes but results still parse.
• ads and related_searches are not included — the Google SERP proxy serves a server-rendered variant where those JS-only modules are absent (verified: the tads ad container arrives empty, "Sponsored" labels never appear). This keeps results clean and ad-free.
• No pagination field — each query returns page 1 results; batch more queries for broader coverage.

Pricing

• $0.005 per completed query (pay-per-event, search-completed)
• Platform usage billed on top, paid by the user: Google SERP proxy ~$0.0025/query + compute ~$0.0015/query (amortized at batch scale)
• Deferred-AIO resolution adds a shell re-fetch + a datacenter proxy call on affected queries only — the datacenter leg is not billed as SERP usage
• All-in ≈ $0.009 per query → 120 queries ≈ $1.05
• vs the leading AI-Overview-only alternative: ~$1.81 for the same 120 queries
• Note: tiny runs (1–5 queries) pay ~$0.02–0.03/query effective due to container cold start — batch ≥25 for best economics

Use cases

• AEO/GEO monitoring — track whether your brand appears in AI Overviews and which sources Google cites, per market with location
• SEO rank tracking at scale
• Research / data collection for LLM training sets
• Competitive SERP intelligence

Costs & limits

• 1000 queries max per run (bump the array; batch in chunks for more)
• Each query = 1 Google SERP request through Apify's Google SERP proxy (+ up to 2 extra requests only when Google defers the AI Overview)
• 10-way concurrency (default): 1000 queries ≈ 30 min (fits the 7200s timeout); rows stream to the dataset as they complete. A/B measured: c5=34s, c10=35s, c20=65s for 20 queries — the SERP proxy pool is the bottleneck, so beyond ~10 workers latency balloons. Set concurrency lower (e.g. 5) for gentler load.
• 7200s timeout (2h), 4096 MB memory
• Build tags: the actor runs the stable build by default (verified, production-hardened). New builds are pushed to latest and promoted to stable only after verification — buyers are never exposed to unverified code.