Google Trends Scraper — Batch Interest Over Time

Slug: google-trends-batch-ppe
Job: Submit 1–N keywords (+ optional geo, timeframe, and category) and receive flat, agent-ready Google Trends interest-over-time rows.

Pay per event: $0.0002 per successful keyword-result ($0.20 per 1,000 successful keyword results). Platform also bills a standard apify-actor-start fee (typically $0.01 per run).

No empty charge: EMPTY, BLOCKED, NO_VOLUME, and FAILED keywords still return status rows, but never create a billable keyword-result event. billed=true is set only when the keyword-result charge succeeds.

Live on Apify Store: https://apify.com/grokbob/google-trends-batch-ppe

Live success is not 100%. Head terms can soft-miss under Google’s limiter; the Actor uses bounded backoff and returns honest BLOCKED/EMPTY/FAILED status rows when data is missing.

⚠️ Residential proxy RETIRED (2026-09-23): Apify RESIDENTIAL sticky-session IP rotate is disabled by default and stripped from the live success path (CapEx-gated pending Bob redesign packet). Do not expect IP rotation on 429/captcha — those fail honestly.

---

Why this Actor

Run batch keyword research without paying for blank answers. This Actor requests Google Trends interest over time one keyword at a time and returns a predictable, flat row for every input keyword.

• Batch-first: submit 1–N keywords with one geo, timeframe, and category configuration.
• Useful trend fields: get avg, peak, latest, and points; opt in to the full timeline when needed.
• Agent-ready output: flat JSON rows are easy to route into agents, Sheets, n8n, Make, and downstream SEO workflows.
• Honest statuses: OK, EMPTY, NO_VOLUME, BLOCKED, and FAILED distinguish a real answer from no data or a request failure.
• Pay for delivered signal: only an OK keyword with a non-empty, non-all-zero interest timeline is billed — and only after the PPE charge succeeds.
• Platform start fee: Apify’s apify-actor-start (about $0.01) applies per run; this Actor does not charge a redundant custom actor-start event.

This is an interest-over-time batch Actor, not a promise of Google Trends trending-now coverage. Related queries and regional fields are optional output features; the live Store price above is for successful keyword-result events.

---

Pricing (live Store pricing)

Item	Price	Billing rule
keyword-result	$0.0002 per successful keyword result ($0.20 / 1,000)	Charged only when the keyword returns a non-empty interest-over-time timeline with at least one real, non-all-zero datapoint and the charge API accepts the event.
apify-actor-start	~$0.01 per run (platform)	Billed by Apify automatically. Do not add a custom actor-start event.
Apify compute	Usage-based	CU appears on the run usage sheet. Residential proxy path is retired (no RESIDENTIAL GB by default).
EMPTY, NO_VOLUME, BLOCKED, or FAILED	$0 keyword-result	A status row is returned for the keyword; no keyword-result event is billed.

Pricing note: primary billable event is keyword-result only. If Console still shows a non-zero apify-default-dataset-item, set it to $0 so customers are not double-charged per dataset row.

---

Inputs

Field	Type	Default	Notes
keywords	string[]	required	1–N keywords / short phrases
geo	string	"" (worldwide)	e.g. US, GB
timeframe	string	today 12-m	Google Trends time tokens
category	integer	0	0 = all
includeRelated	boolean	true	Top related queries on OK rows
includeRegion	boolean	false	Top regions on OK rows
includeTimeline	boolean	false	Embed full timeline array
maxKeywords	integer	50	Hard cap
dryRunMock	boolean	false	Local/CI only — no live Google
useResidentialProxy	boolean	false (retired)	Ignored. RESIDENTIAL sticky rotate stripped; CapEx-gated redesign pending. Explicit true logs a warning and still does not mint residential URLs.
maxAttempts	integer	3	Soft-miss backoff retries on same egress; 429/captcha → honest BLOCKED (no IP rotate)

---

Outputs (dataset)

One row per keyword:

Field	Description
keyword, geo, timeframe	Echo of request
status	OK | EMPTY | NO_VOLUME | BLOCKED | FAILED
billed	true only after successful keyword-result charge
avg, peak, latest, points	Summary stats (OK only)
relatedTop, regionsTop	Optional packs
reason	Present on unbilled / failure rows (explore_no_widgets, multiline_empty, no_timeline_after_N, *_captcha_*, …)
attempts, warmMethod	Retry / warm diagnostics

---

How Trends is fetched (v1)

Path: unofficial Google Trends widget endpoints (same family as pytrends), via got-scraping + CookieJar warm — no Apify residential proxy. Not Playwright by default.

1. Fresh CookieJar per keyword (no Apify residential proxy); warm NID then explore/multiline widgets
2. Warm NID cheaply: GET /_/TrendsUi/data/batchexecute (often 405, still sets NID) → fallback GET /trends/
3. /trends/api/explore → widget tokens (robust extract; prefer TIMESERIES / INTEREST_OVER_TIME)
4. /trends/api/widgetdata/multiline → interest-over-time (~800 ms paced)
5. Optional relatedsearches / comparedgeo widgets
6. On 429/captcha: backoff then honest BLOCKED (no IP rotate). Soft miss: bounded backoff retries up to maxAttempts on same egress — null/empty is not treated as done

Responses strip the )]}' anti-XSSI prefix then JSON-parse. HTTP 200 + captcha/unusual-traffic HTML is classified as BLOCKED with a *_captcha_http_* reason (never a bare warm_http_200).

Known failure modes → status:

Symptom	Status	Billed?
403 / 429 / captcha HTML	BLOCKED	No
Empty timeline after retries	EMPTY (multiline_empty)	No
All-zero values	NO_VOLUME	No
Null timeline after N attempts / missing widget	FAILED (no_timeline_after_N, explore_no_*)	No
≥1 real datapoint + charge OK	OK	Yes (keyword-result)

v1 scope: interest-over-time + related (+ optional region). Trending-now 125 countries deferred.

Playwright: not used in this build. Escalate only with Bob OK / CapEx redesign packet (residential path is retired, not a silent re-enable).

---

Kill criteria (summary)

Stop / pivot if after soft launch:

1. Monitored success rate < 85% for 14 consecutive days under low-friction egress (no residential rotate)
2. Compute COGS > ~40% of PPE revenue for 2 billing cycles
3. Incumbents close the honesty/schema gap and 30d users stay < 50 after 60 days
4. Google breaks the unofficial path with no alternate in 1 sprint
5. Bob’s legal/ToS risk call says stop

Full detail: ../GAP_CARD.md.

---

Local dry smoke (no Google, no spend)

cd google-trends-batch-ppe
npm install
npm test                 # unit/smoke with mocks
ACTOR_TEST_MODE=mock npm start   # full Actor local run with mock fixtures

Mock keywords: normal terms → OK; empty → EMPTY; novolume → NO_VOLUME; blocked → BLOCKED.

---

Bob’s 3 touches before publish

1. Apify account — create / verify publisher account
2. Payout — connect payout method
3. Approve listing — title, PPE event prices, README claims, category tags

Do not publish until Bob says go. Do not spend beyond free-tier test credits without explicit OK. See AGREEMENT.md.