Google AI Overviews Scraper & API ($4/1k) ✨

✨ Google AI Overviews Scraper & AEO API: Clean Markdown, Unwrapped Citations, Brand Visibility Audits & Organic Ranks

---

---

📖 Overview

Google AI Overviews now sit directly above Page 1 organic rankings across informational, commercial comparison, software evaluation, financial, and technical queries — fundamentally reshaping click-through rates and brand discovery.

For SEO agencies, in-house growth teams, and AI data engineers, tracking AI Overviews reliably at scale has been notoriously difficult:

• Truncated "Show More" Answers: Standard SERP scrapers only capture the opening summary paragraph and miss the collapsed comparison tables, nested lists, code snippets, and secondary citation cards that make up ~75% of the full answer.
• Obfuscated Redirect Links: Raw SERP captures often return opaque Google redirect wrappers instead of clean, canonical publisher URLs and root domains needed for attribution dashboards.
• Heavy Browser Costs: Headless-browser scrapers burn 1–4 GB RAM, take 45–90 seconds per keyword, and charge $25.00 to $200.00 per 1,000 queries.
• Disconnected SEO vs. AEO Reporting: Teams have had to run separate tools just to answer a single question: "Do we rank on Page 1 organically, and is Google actually citing our domain inside the AI Overview?"

Google AI Overviews Scraper by UnitBytes delivers the entire expanded AI Overview + unwrapped canonical citations + Page 1 organic rankings + People Also Ask + automated AEO Brand Citation Audit in a single lightweight 256 MB run starting from $4.00 per 1,000 queries.

⚡ UnitBytes · Google AI, Search, News, Trends, Shopping & Local Intelligence Ecosystem
✨ Google AI Overviews Scraper 📍 You are here (From $4.00/1K) SERP AI Overviews, citations, AEO brand audit & Page 1 ranks	🤖 Google AI Mode Scraper From $4.00/1K • Deep AI Answers Full conversational Google AI Mode answers, cited web sources & follow-ups	📈 Google Trends Scraper From $0.30/1K • Search Demand Breakout topics, interest over time, regional heatmaps & rising queries
📰 Google News Scraper From $1.00/1K • PR & Coverage Direct publisher URLs, recency filters & clean full-article text	🛍️ Google Shopping Scraper From $2.00/1K • Retail & Offers Product prices, deals, multi-seller offers & store checkout URLs	📍 Google Maps & Jobs From $0.25/1K • Local & Hiring B2B Leads, Reviews & Google Jobs

---

🌟 Why SEO / AEO Agencies, SaaS Brands & RAG Engineers Choose UnitBytes

Capability / Metric	UnitBytes Google AI Overviews Scraper	Legacy SERP APIs	Browser-Based Scrapers
💰 Price per 1,000 Queries	From $4.00 ($0.0040 / query)	$15.00 – $30.00 per 1K + extra async token calls	$30.00 – $200.00 per 1K + 2–4 GB RAM compute
📝 Full Expanded Answer (markdown)	✅ Complete GFM tables, fenced code blocks, LaTeX math & nested lists	⚠️ Often truncated or requires manual follow-up calls	⚠️ Frequently misses lazy-expanded sections
🔗 Unwrapped Canonical Citation URLs	✅ Direct external https://... URLs + normalized domain	⚠️ Raw redirect wrappers or partial links	⚠️ Redirect links requiring post-processing
🎯 Built-In AEO Brand Citation Audit	✅ brandCitationAudit (cited vs competitor_cited + organicTargetRanks)	❌ Manual scripting required	❌ Not included
📊 Unified SEO + AEO Single-Pass Output	✅ AI Overview + Citations + Page 1 Organic + PAA in 1 item	⚠️ Separate endpoints / extra charges	⚠️ Fragile multi-selector scraping
🌍 Global + Hyper-Local Geo-Targeting	✅ 150+ Countries (country) + City/State (location) + EEA support	✅ Supported	⚠️ Often blocked by European consent screens

---

🎯 Built-In AEO Brand Citation Audit (targetDomains)

Pass your own domains or client domains in targetDomains (for example ["hubspot.com", "pipedrive.com"]) and every output record automatically includes a turnkey brandCitationAudit object cross-referencing AI Overview citations against Page 1 organic rankings:

• citationStatus:
  • "cited" — At least one of your targetDomains is directly cited inside the AI Overview.
  • "competitor_cited" — Google rendered an AI Overview with external citations, and competing domains were cited instead of yours.
  • "overview_no_citations" — Google rendered a synthesized AI Overview with zero external web links.
  • "no_overview" — Google did not display an AI Overview for this query (hasAiOverview: false).
• citedTargetUrls & citedTargetCount: Every exact URL from your target domains cited in the AI Overview.
• topCompetitorDomains: Ranked tally of competing root domains winning AI Overview citations on that keyword.
• organicTargetRanks: Your exact Page 1 organic ranking positions (rank, domain, url) on the same SERP — making it effortless to spot "SEO Rank #2, AEO Cited #0" content gaps!

---

💰 Transparent Pay-Per-Event Pricing (From $4.00 / 1,000 Queries)

This Actor uses transparent Pay-Per-Event (PPE) billing on a minimal 256 MB container footprint — with zero browser compute surcharges and zero hidden secondary-call fees when Google lazy-loads an AI Overview, and $0.00 query charge on genuine zero-result searches.

Apify Plan Tier	Event Name	Price per Query	Price per 1,000 Queries	Included in $5.00 Credit
Free Plan (FREE)	ai-overview-query	$0.0050	$5.00 / 1,000 queries	~995 queries / month free
Paid Volume Plans (GOLD+)	ai-overview-query	$0.0040	$4.00 / 1,000 queries	~1,248 queries per $5.00
Actor Run Start	apify-actor-start	$0.0050	$5.00 / 1,000 runs	One-time startup fee per run (256 MB allocation)

💡 Concrete Cost Examples & Spending-Cap Control

• Free Tier Monthly Audit ($0.00 out-of-pocket): Run a weekly 200-keyword AEO visibility audit (800 queries/month total) completely covered by Apify's free $5.00 monthly platform credit ($0.005 start + 200 × $0.005 = $1.005 per weekly run on Free Tier).
• Agency 10,000-Keyword Multi-Client Tracker (Paid Tier): Auditing 10,000 commercial keywords with full AI Overview Markdown, unwrapped citations, Page 1 organic ranks, PAA, and brandCitationAudit costs just $40.005 ($4.00 / 1,000).
• Hard Budget Cap Support: Set Maximum cost per run in the Apify Console (Options -> Maximum cost per run) or via API (maxTotalChargeUsd). The Actor monitors remaining budget in real time and stops cleanly with exit code 0 as soon as your spending cap is reached.

---

✨ Key Capabilities

• 📝 LLM-Ready GitHub-Flavored Markdown (markdown) + Structured AST (textBlocks): Reconstructs complete headings (###), bold callouts, nested bulleted and numbered lists, multi-column comparison tables (| Feature | Option A | Option B |), syntax-tagged code blocks (```python), and inline/block LaTeX formulas ($O(\log n)$).
• 🔗 Unwrapped Canonical External Citations (citations): Resolves citation cards into clean external URLs (https://...), normalized root domain, publisher, title, snippet, and faviconUrl.
• 📊 Side-by-Side Page 1 Organic Rankings (organicResults): Captures traditional Page 1 search rankings (rank, title, url, domain, displayedUrl, snippet) in the exact same pass so you can correlate classic SEO rank with AI Overview citation share.
• ❓ People Also Ask (peopleAlsoAsk): Extracts related user follow-up questions surfaced on the SERP for content brief generation.
• 🌍 150+ Countries & City-Level Geo-Targeting (country + location): Audit national markets (US, GB, DE, FR, CA, AU, IN, BR, JP, ES, IT, NL, SG, AE, etc. — including full European EEA support) or pinpoint exact cities ("Austin, Texas, United States", "Manchester, England, United Kingdom").

---

🔍 When hasAiOverview Is false (Honest Upstream SERP Behavior)

Google does not generate an AI Overview on 100% of searches. In real-world SERPs, hasAiOverview is false (aiOverviewState: "none") in three common scenarios:

1. Pure Navigational or Operator Queries: Brand login searches, exact domain lookups, or search operators such as site:example.com frequently bypass AI Overviews upstream.
2. Commercial / YMYL Queries Without AI Trigger: On certain transactional or localized queries, Google intentionally serves traditional organic listings only.
3. SERP Recency Filters (timeRange != "any"): Applying publication-date filters (hour, day, week, month, year) restricts Google's organic index by recency and causes Google upstream to suppress AI Overviews on most queries. Keep timeRange: "any" whenever your primary goal is capturing AI Overviews.

Why Every Query Still Delivers Full SERP Value: Even when Google suppresses the AI Overview (hasAiOverview: false), this Actor still returns the query's Page 1 organicResults, peopleAlsoAsk, and brandCitationAudit (citationStatus: "no_overview" + your organicTargetRanks), giving your SEO/AEO reporting pipeline complete visibility into both AI-triggered and classic SERPs without blind spots.

---

🛠️ How to Use via API (Python, Node.js & cURL)

🐍 Python Example (AEO Brand Visibility & Competitor Citation Audit)

$pip install apify-client

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_API_TOKEN")

run_input = {
    "queries": [
        "best crm for small business",
        "compare hubspot vs pipedrive vs zoho for 10 person sales team",
        "difference between postgresql and mysql",
    ],
    "country": "US",
    "targetDomains": ["hubspot.com", "pipedrive.com"],
    "includeOrganicResults": True,
    "includePeopleAlsoAsk": True,
    "resolveUrls": True,
    "maxQueries": 25,
}

run = client.actor("unitbytes/google-ai-overviews-scraper").call(run_input=run_input)
items = client.dataset(run["defaultDatasetId"]).list_items().items

for row in items:
    audit = row.get("brandCitationAudit") or {}
    print(
        f"[{row['country']}] {row['query']} | AIO={row['hasAiOverview']} "
        f"| Citations={row['citationsCount']} | Audit={audit.get('citationStatus')}"
    )

---

🟨 JavaScript / Node.js Example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_API_TOKEN' });

const run = await client.actor('unitbytes/google-ai-overviews-scraper').call({
    queries: [
        'best roofing contractors near me',
        'roof replacement cost estimator',
    ],
    country: 'US',
    location: 'Austin, Texas, United States',
    includeOrganicResults: true,
    resolveUrls: true,
    maxQueries: 10,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`Processed ${items.length} localized SERP + AI Overview records`, items[0]);

---

🌐 cURL Example

curl -X POST "https://api.apify.com/v2/acts/unitbytes~google-ai-overviews-scraper/runs?token=YOUR_APIFY_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": ["how does a stocks and shares isa work in the uk"],
    "country": "GB",
    "targetDomains": ["gov.uk", "moneysavingexpert.com"],
    "includeOrganicResults": true,
    "resolveUrls": true,
    "maxQueries": 5
  }'

---

📤 Example Output Record

{
  "query": "compare hubspot vs pipedrive vs zoho for 10 person sales team",
  "country": "US",
  "language": "en",
  "location": null,
  "timeRange": "any",
  "hasAiOverview": true,
  "aiOverviewState": "deferred",
  "markdown": "Comparing **HubSpot**, **Pipedrive**, and **Zoho CRM** for a 10-person sales team depends on whether you prioritize ease of use, marketing automation depth, or cost-effective customization.\n\n### Quick Comparison at a Glance\n\n| Feature | HubSpot Sales Hub | Pipedrive | Zoho CRM |\n| --- | --- | --- | --- |\n| Best For | All-in-one GTM & marketing alignment | Visual pipeline & outbound velocity | Budget-friendly customization |\n| Learning Curve | Low to Moderate | Very Low | Moderate |\n| Reporting | Advanced multi-touch attribution | Deal velocity & activity tracking | Custom BI modules & forecasting |",
  "text": "Comparing HubSpot, Pipedrive, and Zoho CRM for a 10-person sales team depends on whether you prioritize ease of use, marketing automation depth, or cost-effective customization...",
  "textBlocks": [
    {
      "type": "paragraph",
      "text": "Comparing HubSpot, Pipedrive, and Zoho CRM for a 10-person sales team depends on whether you prioritize ease of use, marketing automation depth, or cost-effective customization."
    },
    {
      "type": "heading",
      "level": 3,
      "text": "Quick Comparison at a Glance"
    },
    {
      "type": "table",
      "headers": ["Feature", "HubSpot Sales Hub", "Pipedrive", "Zoho CRM"],
      "rows": [
        ["Best For", "All-in-one GTM & marketing alignment", "Visual pipeline & outbound velocity", "Budget-friendly customization"]
      ]
    }
  ],
  "citations": [
    {
      "index": 1,
      "title": "HubSpot vs. Pipedrive vs. Zoho CRM Comparison",
      "url": "https://www.pcmag.com/picks/the-best-crm-software",
      "domain": "pcmag.com",
      "publisher": "PCMag",
      "snippet": "Pipedrive focuses squarely on sales pipeline management, while HubSpot unifies marketing, sales, and service hubs.",
      "faviconUrl": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.pcmag.com&sz=64"
    }
  ],
  "citationsCount": 1,
  "brandCitationAudit": {
    "targetDomains": ["hubspot.com", "pipedrive.com"],
    "citationStatus": "competitor_cited",
    "isCited": false,
    "citedTargetUrls": [],
    "citedTargetCount": 0,
    "topCompetitorDomains": [
      { "domain": "pcmag.com", "citationsCount": 1 }
    ],
    "organicTargetRanks": [
      {
        "rank": 2,
        "domain": "hubspot.com",
        "url": "https://www.hubspot.com/comparisons/pipedrive-vs-hubspot"
      }
    ]
  },
  "organicResults": [
    {
      "rank": 1,
      "title": "The Best CRM Software for Small Businesses",
      "url": "https://www.pcmag.com/picks/the-best-crm-software",
      "domain": "pcmag.com",
      "displayedUrl": "https://www.pcmag.com › Business › CRM",
      "snippet": "We tested the top CRM platforms for small and midsize sales teams..."
    },
    {
      "rank": 2,
      "title": "Pipedrive vs. HubSpot: Which CRM Fits Your Sales Team?",
      "url": "https://www.hubspot.com/comparisons/pipedrive-vs-hubspot",
      "domain": "hubspot.com",
      "displayedUrl": "https://www.hubspot.com › comparisons",
      "snippet": "Compare HubSpot Sales Hub and Pipedrive side by side across pipeline automation, reporting, and seat pricing."
    }
  ],
  "organicResultsCount": 2,
  "peopleAlsoAsk": [
    {
      "question": "Is Pipedrive or HubSpot better for a 10-person team?",
      "answer": "Pipedrive is often faster to onboard for pure outbound sales teams, whereas HubSpot scales better if marketing and sales share attribution.",
      "title": "CRM Buyer Guide",
      "url": "https://www.pcmag.com/picks/the-best-crm-software",
      "domain": "pcmag.com"
    }
  ],
  "googleSearchUrl": "https://www.google.com/search?q=compare+hubspot+vs+pipedrive+vs+zoho+for+10+person+sales+team&gl=us&hl=en",
  "scrapedAt": "2026-11-14T12:00:00Z"
}

Complete Output Fields Reference (18 Root Fields)

Field	Type	Description
query	string	Normalized Google Search query processed.
country	string	Uppercase 2-letter ISO 3166-1 country code ("US", "GB", "DE", "JP", etc.).
language	string	Effective interface language code used ("en", "de", "fr", "pt-BR", "ja", etc.).
location	string | null	Canonical city/region geo-targeting location string if supplied, otherwise null.
timeRange	string	Applied SERP recency filter ("any", "hour", "day", "week", "month", "year").
hasAiOverview	boolean	true when Google rendered an AI Overview on this SERP.
aiOverviewState	string	"inline" (rendered directly on initial SERP), "deferred" (expanded dynamically), "none", or "no_results".
markdown	string	Full AI Overview answer in clean GitHub-Flavored Markdown (headings, tables, lists, code, math).
text	string	Clean plain-text version of the AI Overview answer.
textBlocks	object[]	Structured block list (paragraph, heading, list, table, code) for deterministic RAG chunking.
citations	object[]	Unwrapped source citations (index, title, url, domain, publisher, snippet, faviconUrl).
citationsCount	integer	Total number of citations in citations.
brandCitationAudit	object | null	AEO audit object (targetDomains, citationStatus, isCited, citedTargetUrls, citedTargetCount, topCompetitorDomains, organicTargetRanks) when targetDomains is set.
organicResults	object[]	Page 1 organic search results (rank, title, url, domain, displayedUrl, snippet).
organicResultsCount	integer	Count of Page 1 organic results extracted.
peopleAlsoAsk	object[]	Related People Also Ask entries (question, answer, title, url, domain).
googleSearchUrl	string	Direct Google Search URL for the query and market.
scrapedAt	string	UTC ISO 8601 extraction timestamp.

---

⚙️ Input Parameters Reference

Parameter	Type	Default	Description
queries	string[]	["best crm for small business", ...]	Search keywords, questions, or direct Google Search URLs. Automatically deduplicated.
startUrls	object[]	[]	Optional Google Search URLs ({"url": "https://www.google.com/search?q=..."}).
maxQueries	integer	25	Maximum unique queries to process in this run (1 – 1000).
country	string	"US"	2-letter ISO 3166-1 country code (US, GB, DE, FR, CA, AU, IN, BR, JP, etc.).
language	string	""	Optional ISO/BCP-47 language override. Leave empty ("") to automatically use the natural language of country.
location	string	""	Optional city/state/region geo-targeting (e.g. "Austin, Texas, United States").
timeRange	string	"any"	"any", "hour", "day", "week", "month", or "year". Note: non-"any" recency filters apply to organic SERPs and generally suppress AI Overviews upstream.
targetDomains	string[]	[]	Brand/client domains (e.g. ["hubspot.com"]) to populate brandCitationAudit.
includeOrganicResults	boolean	true	Include Page 1 traditional organic search rankings.
includePeopleAlsoAsk	boolean	true	Include People Also Ask questions on the SERP.
resolveUrls	boolean	true	Unwrap Google redirect links into clean canonical external URLs.
proxyConfiguration	object	null	Optional custom proxy override (proxyUrls). Managed residential proxies are configured automatically on Cloud.

---

📋 Copy-and-Paste Directive for AI Coding Assistants (Cursor, Windsurf & Claude Code)

"Use the unitbytes/google-ai-overviews-scraper Actor on Apify to extract Google Search AI Overviews in clean Markdown, unwrapped source citations, Page 1 organic rankings, People Also Ask, and brandCitationAudit (targetDomains). Pair it with unitbytes/google-ai-mode-scraper for deep conversational Google AI Mode benchmarking. Example call: client.actor('unitbytes/google-ai-overviews-scraper').call(run_input={'queries': ['best crm for small business'], 'country': 'US', 'targetDomains': ['hubspot.com'], 'maxQueries': 25})"

---

🔌 MCP Server Setup: Claude Code, Cursor & AI Agents

{
  "mcpServers": {
    "apify": {
      "type": "http",
      "url": "https://mcp.apify.com/?tools=actors,docs,unitbytes/google-ai-overviews-scraper,unitbytes/google-ai-mode-scraper"
    }
  }
}

---

⚖️ Compliance & Privacy

This Actor extracts strictly publicly available search results and AI Overview summaries displayed on Google Search. It never accesses private Google accounts, personal histories, or personally identifiable information (PII). Read Apify's guide on the legality of web scraping to ensure compliance with applicable regulations.

Disclaimer

This Actor is an independent software tool and is not affiliated with, endorsed by, or sponsored by Google LLC.

---

💬 Support & Feedback

Building an AEO visibility dashboard, multi-market SEO tracker, or enterprise RAG pipeline?

• 💬 Apify Issues: Open a ticket on the Actor Issues Tab
• 🚀 Apify Store: Google AI Overviews Scraper