Google SERP & AEO Scraper

Apify Actor for automated Google Search scraping with GOOGLE_SERP proxy, manual search operators, country targeting, ads detection, flat SEO export, and optional LLM visibility comparison.

Differentiator: includes a Google Sheets workflow — run SERP + AEO scans from a spreadsheet (keywords, settings, results, run cost) with no coding after a 5-minute setup.

Google Sheets workflow

Run this Actor from Google Sheets — most users never touch the API.

Copy template (fastest)	Make a copy
Install guide	morph-coder/google-serp-aeo-scraper

Quick start

1. Activate the Actor in your Apify account.
2. Copy the template spreadsheet → SERP Tools → Configure Apify token.
3. Add keywords on the Keywords sheet → SERP Tools → Run SERP scan → results land in Results, LLM Summary, and Run Log (costUsd per run).

Prefer your own spreadsheet? Paste Code.gs via Extensions → Apps Script — step-by-step in the Sheets install guide. Review the script with your AI assistant before authorizing Google permissions.

Screenshots

Results — flat SERP rows (rank, URL, resultsTotal, LLM citation flags):

LLM Summary — visibility overlap and ranks per keyword:

Run Log — organic counts, LLM calls, and costUsd per run:

Features

• Google SERP via Apify GOOGLE_SERP proxy + CheerioCrawler
• Search operators: site:, related:, intitle:, date filters, file types, exact match
• Country / language / UULE geo targeting (countryCode → google.{tld} + GOOGLE_SERP proxy country, with fallback to global SERP pool)
• Pagination: maxResultsPerQuery (per keyword), maxTotalResults (whole-run cap), or maxPagesPerQuery (fixed pages). See Limit priority below.
• Output formats: page (Apify-compatible), flat (Sheets/Zapier), both
• Paid ads + shopping ads parsing, PAA, related queries, AI Overview
• Optional LLM add-ons (API): ChatGPT, Gemini, Perplexity, DeepSeek, Copilot
• Google Sheets — template + Apps Script (workflow)

Input example

Single keyword, top-20:

{
  "queries": ["NASA"],
  "countryCode": "us",
  "maxResultsPerQuery": 20,
  "outputFormat": "flat"
}

Multiple keywords (one dataset, filter by keyword):

{
  "queries": ["NASA", "SpaceX", "Tesla"],
  "countryCode": "us",
  "maxResultsPerQuery": 20,
  "outputFormat": "flat"
}

→ up to 60 organic rows (20 × 3 keywords). Leave maxTotalResults unset for full per-keyword depth.

Limit priority

When several limits are set, they apply in this order:

Priority	Setting	Scope
1	maxTotalResults	Hard cap on all keywords combined. When set, keywords run one by one; the first keyword(s) consume the budget.
2	maxResultsPerQuery	Target top-N per keyword (may be cut short if global cap runs out).
3	maxPagesPerQuery	Used only when maxResultsPerQuery is not set.
—	Empty SERP page (organicOnCurrentPage === 0)	No more rows — stop even if start= could advance.
—	Missing Google "Next" link	Not a stop when maxResultsPerQuery is set — actor continues via start= until target or empty page.

Example — 2 keywords, maxResultsPerQuery: 20, maxTotalResults: 34:

nike   → 20 rows (uses 20 of 34 global budget)
adidas → 14 rows only (34 − 20 = 14 left; stops before top-20)

To get 20 + 20 = 40 rows, either omit maxTotalResults or set it ≥ 40 (e.g. queries.length × maxResultsPerQuery).

The run ends with usage_summary.billing.organicByKeyword so you can see per-keyword counts vs limits.

Cap total billing with maxTotalResults (optional):

{
  "queries": ["kw1", "kw2", "kw3", "kw4", "kw5", "kw6", "kw7", "kw8", "kw9", "kw10"],
  "maxResultsPerQuery": 20,
  "maxTotalResults": 20,
  "outputFormat": "flat"
}

→ 20 rows total (first keyword(s) until cap), not 200.

Legacy API: queries may also be a newline-separated string.

Output

Page record — one row per SERP page with organicResults[], paidResults[], etc.

Flat record (recordType: flat) — one row per SERP result:

keyword, resultsTotal (Google estimate, e.g. About 1,230,000,000 results — same on all rows for that keyword), rank, serpSlot, queryPage, position, title, url, type, isPaid, targetCountry, llm*Cited, llm*UrlRank, …

keyword	rank	serpSlot	queryPage	position	title	url	isPaid

LLM record (recordType: llm) — one row per keyword/page when AI add-ons are enabled and outputFormat is flat. Contains chatGptSearchResult, geminiSearchResult, visibilityCompare, etc. Use dataset view LLM answers or ?view=llm_results.

With outputFormat: page or both, LLM fields live on the page record instead.

• resultsTotal — Google’s estimated match count from the SERP header (About X results). Parsed from page 1 and copied to all flat rows for that keyword. Approximate — not the same as rows you scrape.
• rank — sequential 1…N for analytics (top-20, charts). No gaps.
• serpSlot — estimated Google slot (page−1)×10+position; may skip numbers when a page has <10 organic results.
• queryPage + position — where on the SERP page the result appeared.

API export (no flat rows needed):

GET /v2/datasets/{id}/items?format=json&fields=searchQuery,organicResults&unwind=organicResults

Local development

npm install
npm run build
cp .env.example .env   # optional
npx apify run --input scripts/test-input.example.json

Pay-per-event billing

Pricing model: Pay per event (active from June 15, 2026). Dataset rows are free — apify-default-dataset-item = $0.

Apify displays prices per 1,000 events in the Console. Below are the published rates from Monetization settings.

Platform usage: User pays platform usage costs (proxy, compute, memory) on top of PPE events. usageTotalUsd in Run Log includes both.

Competitive context

Actor / tier	Unit	Effective $ / 1,000 URLs*
apify/google-search-scraper	$2.50 / 1,000 pages	~$0.25 (≈10 organic/page)
Budget scrapers on Store	per URL	~$0.05–0.15
This Actor (organic, base tier)	per delivered URL	$0.40
This Actor (Bronze Apify plan)	per delivered URL	$0.30

*Example: 50 keywords × top-20 = 1,000 URLs → Apify scraper ≈ $0.25 SERP; us $0.40 (base) or $0.30 (Bronze) + platform usage.

Model	Apify official scraper	This Actor
SERP unit	Per page fetched	Per organic URL delivered
Sparse page (6 results)	Full page price	Only rows you get
Output	SERP JSON	Flat rows + optional LLM citations + visibility compare

Organic pricing uses Apify plan tier discounts (Bronze / Silver / Gold). LLM add-ons are flat per provider.

Event prices (Console)

Event	Price / 1,000	Per unit	Notes
organic-result-scraped	$0.40 (base)	$0.0004 / URL	Tiered — see below
↳ Bronze discount	$0.30	$0.0003 / URL	Apify Bronze plan
↳ Silver discount	$0.28	$0.00028 / URL	Apify Silver plan
↳ Gold discount	$0.26	$0.00026 / URL	Apify Gold plan
chatgpt-search-scraped	$25.00	$0.025 / call	Successful web search only
gemini-search-scraped	$25.00	$0.025 / call
perplexity-search-scraped	$30.00	$0.03 / call
copilot-search-scraped	$25.00	$0.025 / call
deepseek-search-scraped	$20.00	$0.02 / call	Experimental — chat only, no web citations
ads-scraped	$0.25	$0.00025 / ad	Only when focusOnPaidAds: true
apify-actor-start	—	$0.0005 / run	Actor start event

Failed LLM calls (missing API key, API error) are not charged.

Pricing examples

PPE only (base organic tier). Add platform usage on top — see usageTotalUsd in Run Log.

Bulk SEO — 50 keywords, top-20 organic, SERP only (1,000 flat rows)

1 × actor start        $0.0005
1000 × organic        $0.4000
──────────────────────────────
PPE subtotal          ≈ $0.40 / run     (Bronze organic ≈ $0.30; Apify scraper ≈ $0.25)
+ platform usage

1 keyword, top-20 organic, ChatGPT + Gemini + Perplexity + DeepSeek (~3 SERP pages fetched)

1 × actor start              $0.0005
20 × organic                 $0.0080
1 × ChatGPT                  $0.0250
1 × Gemini                   $0.0250
1 × Perplexity               $0.0300
1 × DeepSeek                 $0.0200
──────────────────────────────────────
PPE subtotal                 ≈ $0.11 / run
+ platform usage

Light — 1 keyword, top-10 organic, ChatGPT only

1 × actor start     $0.0005
10 × organic       $0.0040
1 × ChatGPT          $0.0250
─────────────────────────
PPE subtotal       ≈ $0.03 / run
+ platform usage

Typical — 2 keywords, top-10 organic, ChatGPT

1 × actor start     $0.0005
20 × organic       $0.0080
2 × ChatGPT          $0.0500
─────────────────────────
PPE subtotal       ≈ $0.06 / run
+ platform usage

SEO batch — 5 keywords, top-20 organic, ChatGPT

1 × actor start     $0.0005
100 × organic      $0.0400
5 × ChatGPT          $0.1250
─────────────────────────
PPE subtotal       ≈ $0.17 / run
+ platform usage

Formula (PPE only, base organic tier):

PPE cost ≈ $0.0005
         + (organic URLs × organic rate)     // $0.0004 base; $0.0003 Bronze; etc.
         + Σ(LLM calls × provider rate)      // ChatGPT/Gemini/Copilot $0.025; Perplexity $0.03; DeepSeek $0.02
         + (paid ads × $0.00025)             // if focusOnPaidAds
         + platform usage                    // proxy + compute (billed separately)

Margin notes (for Actor owner)

Item	~API cost	PPE price (base)	~margin
Organic URL	proxy/compute (platform → user)	$0.0004	PPE margin; platform passed through
ChatGPT / Gemini / Copilot	~$0.01/call	$0.025	~150%
Perplexity	~$0.01/call	$0.03	~200%
DeepSeek	~$0.01/call	$0.02	~100%
Dataset rows	—	$0	Sheets workflow

usageTotalUsd in Run Log (Sheets) and Apify Console is the source of truth for what the user paid (PPE + platform). API dashboards track your LLM COGS separately.

Google SERP — per organic URL

Event	When	Count
organic-result-scraped	Each organic result row saved	1 per URL

You pay for delivered organic URLs, not for SERP pages fetched. Capped by maxResultsPerQuery and maxTotalResults.

Estimate:

SERP PPE ≈ organic URLs × your tier rate   // base $0.0004; Bronze $0.0003; Silver $0.00028; Gold $0.00026
           + platform usage

Paid ads (optional)

Event	When
ads-scraped	Each paid ad row when focusOnPaidAds: true

LLM add-ons — per API call

Event	When	Count
chatgpt-search-scraped	Successful ChatGPT web search	1 per keyword (perQuery) or per SERP page (perPage)
gemini-search-scraped	Successful Gemini call (with citations)	same
perplexity-search-scraped	Successful Perplexity call	same
deepseek-search-scraped	Successful DeepSeek chat completion	same — experimental
copilot-search-scraped	Successful Copilot call	same
ai-mode-result-scraped	AI Mode add-on	not implemented — do not charge

LLM billing is per request, not per citation. Failed calls are not charged.

Estimate:

LLM PPE ≈ Σ(successful calls × provider rate)
         // ChatGPT / Gemini / Copilot $0.025; Perplexity $0.03; DeepSeek $0.02

Run audit row

The dataset ends with recordType: usage_summary:

{
  "billing": {
    "organicUrlsCharged": 17,
    "llmCallsCharged": { "chatGpt": 2 },
    "pagesFetched": 4
  },
  "llmTokenUsage": { "...": "provider token totals for your margin" }
}

pagesFetched is informational only — not used for client billing.

LLM secrets (Actor environment variables)

Variable	Provider
OPENAI_API_KEY	ChatGPT (Responses API + web_search)
GEMINI_API_KEY	Gemini
PERPLEXITY_API_KEY	Perplexity
DEEPSEEK_API_KEY	DeepSeek
AZURE_OPENAI_*	Copilot

ChatGPT uses OpenAI Responses API with the hosted web_search tool. Gemini uses google_search grounding on the Generative Language API (gemini-2.5-flash by default). Set OPENAI_MODEL / GEMINI_MODEL in Actor secrets.

DeepSeek (experimental): plain chat completion only — lowest API token cost, but no web search or citation URLs yet. Useful for testing narrative/brand mentions; not recommended for AEO rank tracking until a search API is added. Priced at $0.02/call ($20 / 1,000) in Console.

LLM input options

Field	Default	Purpose
llmQueryMode	sameAsKeyword	Send exact Google keyword to LLM web search
llmApplyTo	brandLike	Skip commercial/long-tail keywords for LLM
llmSearchScope	perQuery	One LLM call per keyword (page 1)

visibilityCompare.rankedComparison maps googleRank vs llmRank per URL. rankedComparisonByDomain uses the best rank per domain (handles different paths and utm_* params). targetDomainRanks defaults to Google #1 domain when targetDomains is not set.

Each LLM result includes usage (inputTokens, outputTokens, totalTokens, optional webSearchCalls) and model for your internal margin tracking. See usage_summary.billing for PPE totals per run.

Clients do not pass API keys in input — LLM API cost is included in LLM PPE events.

Google Sheets (more detail)

The Google Sheets workflow section at the top covers copy-template setup. Additional notes:

• Apify token is stored in Script Properties, not in cells — sent only to api.apify.com.
• Results / LLM sheets refresh each run; Run Log is append-only history with costUsd (usageTotalUsd from Apify).

Sheet	Contents
Results	Flat organic SERP + LLM citation columns per URL
LLM Summary	Visibility KPIs + citation counts per provider
LLM Answers	Full LLM answer text (one row per keyword × provider)
LLM Citations	Parsed citations (rank, url, title)
Run Log	Organic counts, LLM calls, costUsd per run

Install from scratch: integrations/google-sheets/README.md.

Disclaimer

Use responsibly. Comply with Google Terms of Service and applicable laws.