๐๏ธ Google Events Scraper extracts event listings from Googleโdates, locations, titles & detailsโfast and accurate. ๐ Perfect for lead gen, market research, and building event databases. ๐ Save time, scale up.
All notable changes to this actor are documented here.
[1.0] - 2026-08-29
Fixed - image field always empty
Investigated live (via a real proxy fetch of the actor's own current output
plus a Chrome DevTools inspection to find the true dedicated ibp=htl;events
vertical page, confirmed still dead for every query tried, including ones
matching the requester's own real location) why several output fields were
always blank. Two different, unrelated findings:
venue.rating/venue.reviews: confirmed genuinely absent, not a bug.
The "Rated X out of 5" markup that does appear elsewhere on the page
belongs to an unrelated business/local-panel snippet, not the event card
itself โ verified by locating its exact DOM position. Left as null,
correctly.
image: real data was available and simply never extracted. Each
card's <img> tag only ever carries a 1x1 GIF placeholder
(data-deferred="1") โ the real photo is assigned elsewhere on the page,
in one of two forms: a JSON-style
mapping to a small permanent-ish CDN thumbnail URL (used
when available โ confirmed live to cover every card), or, as a fallback,
an inline
var s='data:image/...';var ii=['dimg_<id>',...];_setImagesSrc(ii,s)
base64 blob a real browser paints as an immediate
low-res placeholder (one blob can cover several ids โ Google reuses a
generic icon for photo-less events).
Fix (extracto.py): added extract_image_map(), which reads both forms
statically (no browser needed) into one id -> image map built once per
page, and parse_event_card() now resolves each card's real image through
it. link, venue.rating/reviews, and the map thumbnail (a genuinely
separate asset from the event's own photo) remain honestly empty โ confirmed
not present in this card format at all, not merely unextracted.
Verified against a live fetch (same GOOGLE_SERP proxy path production uses):
10/10 events resolved a real, working CDN thumbnail URL.
[0.9] - 2026-08-29
Fixed GOOGLE_SERP for real โ needs plain HTTP, not HTTPS
0.8's hostname fix (bare google.com) was the wrong half of the problem.
Pulled the proxy's own error text directly from the wire (raw CONNECT to
proxy.apify.com:8000, bypassing curl_cffi's truncated error): the real,
first blocker is
"The proxy group "GOOGLESERP" cannot be used over HTTPS connection"
โ it cannot terminate a CONNECT/TLS tunnel at all, only plain
HTTP (it fetches Google over HTTPS server-side and hands back the
rendered page). Retesting confirmed the second half was real too: with
plain HTTP but the 0.8 bare google.com host, the proxy rejects with
"The hostname does not correspond to a supported Google service" โ it
specifically wants www.google.com.
Fix.TIER_GOOGLE_SERP now requests http://www.google.com/search
(HTTP, www. subdomain) instead of the HTTPS URL every other tier uses,
and skips the cookie warm-up step entirely for this tier (irrelevant for a
proxy that renders server-side). Verified with a raw manual request through
this exact proxy/account: HTTP 200, a real 563 KB SERP with id="rso"
and 3 NCvvle event cards present.
. Per Apify's docs, the proxy recognises Google
traffic by hostname prefix match on "google." โ but every request in this
actor went to www.google.com, which starts with "www.", not
"google.", so the proxy's own routing rejected the CONNECT before it ever
reached Google. Fixed by switching SEARCH_URL/HOMEPAGE_URL/Referer to
the bare google.com (no www.), which Google serves identically.
[0.7] - 2026-08-29
The actual fix: use Apify's GOOGLE_SERP proxy group
0.6 pinned down why every plain-HTTP fix still failed: it's IP reputation
specific to Google's search-results endpoint on Apify's generic
datacenter/residential proxy pools, not a request-shape bug. That pointed
straight at the real answer โ Apify sells a proxy group built specifically
for this: GOOGLE_SERP (auto-detects google.* hostnames, no
session_id, priced per request). This is almost certainly what the actor
actually used during its working year, not the generic pools this rewrite
had been falling back to.
Fix. Added TIER_GOOGLE_SERP to the ladder, tried right after a cheap
direct attempt and before datacenter/residential:
direct โ ๐ Google SERP โ ๐ข datacenter โ ๐ residential
. build_proxy_configs() now also requests
groups=["GOOGLE_SERP"]; if that add-on isn't enabled on the account it
just fails to provision and the ladder falls through to datacenter/
residential exactly as before, so this is a strict improvement with no new
failure mode.
[0.6] - 2026-08-29
Diagnostic result: confirmed IP/endpoint-reputation, not a request-shape bug
0.5's warm-up fix was tested live with cookie-jar visibility added
(resp.status_code, len(resp.cookies) logged per tier). Result, at every
one of the three proxy tiers on the same run: the homepage warm-up GET
succeeds cleanly โ HTTP 200, 8 cookie(s) set โ every single time. The very
next request, to /search, on the same session, same cookies, same IP,
is still blocked (JS-required wall at direct/datacenter, HTTP 429 at
residential after 3 attempts).
That homepage-passes-but-/search-fails split is the key evidence: it rules
out a generic "this IP looks like a bot" block (a generic block would also
catch the homepage) and rules out a request-shape/header/cookie defect (the
warmed-up request has real cookies, headers, and TLS impersonation and still
fails identically). What's left is a defense specific to Google's search
results endpoint keyed on the requesting IP's history โ /search is by
far the most heavily automated-hit URL on the internet (every rank tracker,
SEO tool, and SERP scraper hits it), so shared proxy-pool IPs (Apify's
datacenter pool, and even a supposedly-fresh residential-tier IP drawn from
Apify's shared pool) accumulate query-volume reputation against that one
endpoint from other tenants' traffic, independent of anything this actor
does. Proactive CONSENT/SOCS cookies (also added in 0.6, since they're
free and correct to send either way) did not change the outcome either.
Kept: the 0.5 warm-up-for-cookies + explicit headers + consent cookies โ
legitimate hardening, and the only way to be sure a plain "cookie-less
request" theory was actually ruled out rather than assumed.
Not fixed by code, and can't be: getting past a shared-proxy-pool IP's
accumulated reputation against Google's specific search-results endpoint is
an infrastructure problem, not a parsing/header/cookie one. Options that
would actually move this: a proxy pool dedicated to SERP scraping (e.g. a
provider selling IPs specifically maintained for Google Search access,
priced accordingly) instead of Apify's general-purpose residential pool; or
a much lower request rate against a single long-lived IP so it can build
its own clean reputation over time instead of drawing a fresh, already-worn
IP from the shared pool on every run.
[0.5] - 2026-08-29
Real fix attempt #2 โ cold cookie-less request, not a proxy problem
User pushback (correct): this actor ran fine for a year, so the block seen
in 0.4's verification run isn't fundamentally a proxy-quality problem โ
something in this rewrite's request shape itself is what's new. The
rewritten transport hits GET /search stone cold on every single request:
no prior cookies (no NID/AEC/1P_JAR-style trackers a real browser always
carries after ever visiting google.com once), no Accept/Accept-Language/
Referer headers. That's a distinguishable, catchable signature regardless
of TLS/JA3 impersonation or which proxy tier serves it.
Fix. Added warm_up(): one GET to https://www.google.com/ per proxy
tier (best-effort, cookies persist on the shared AsyncSession) before ever
hitting /search, plus explicit Accept/Accept-Language/Referer
headers on every request. This is the same warm-up-for-cookies pattern that
fixed the analogous "always blocked on the very first request" bug in the
Ebay Scraper and eBay Review Scraper actors today.
[0.4] - 2026-08-29
Reverted the 0.3 headless-browser fallback tier โ it crash-looped the actor
0.3 added a 4th proxy-ladder tier (real headless Chromium via Playwright,
routed through the residential proxy) to get past Google's JS-required
interstitial on Apify's shared proxy IPs โ plausible in theory, but a live
Apify verification run showed the container crash-looping
("Running under LIMITED_PERMISSIONS" โ immediate restart, repeated, then
"unexpected system error") before Python even started. Root cause: the
apify/actor-python-playwright base image wraps every command in
xvfb-entrypoint.sh (spawns a virtual X server via xvfb-run), and that
does not start under Apify's LIMITED_PERMISSIONS container sandbox. A
crash-looping actor is strictly worse than an occasional 0-event run, so
0.4 reverts to the plain apify/actor-python:3.13 image and the 3-tier
ladder (direct โ datacenter โ residential), dropping playwright from
requirements.txt.
What's kept from 0.3:build_params() still defaults hl/gl to
en/us when the caller leaves them blank โ a real, low-risk improvement
against Google's separate EU/EEA consent interstitial.
Known, accepted limitation: Google's classic /search can still serve
its JS-required interstitial to plain HTTP even at the residential proxy
tier, on some fraction of runs, specifically on Apify's shared proxy IP
pool (the identical request from an unflagged residential home IP passes
every time with no browser needed). When that happens the actor now fails
closed and cleanly reports 0 events for that query rather than crashing โ
this is a real constraint of scraping Google's core search product from
shared/datacenter-adjacent IPs, not a parsing bug. If this actor needs to be
reliable at higher volume, the real fix is a headless-browser fetch running
in its own dedicated Apify Actor (not this one) so its sandbox/xvfb needs
don't conflict with this actor's LIMITED_PERMISSIONS runtime, or a
higher-quality dedicated residential/mobile proxy pool.
[0.3] - 2026-08-29 (reverted in 0.4, kept for history)
Fixed - still 0 results when actually run on Apify (live verification)
The 0.2 fix (see below) was verified locally against a live Google response
and worked โ but a live Apify-hosted verification run still returned 0
events at every proxy tier, including residential. Root cause: enabled
Actor.log debug capture of the "blocked" page showed Google is serving its
JS-required interstitial (httpservice/retry/enablejs) even at the
residential tier, 100% of the time, regardless of TLS/JA3 impersonation โ
this is specific to Apify's shared proxy IP pool (the same request from an
unflagged residential home IP passes with plain HTTP, no browser needed).
Plain-HTTP transport alone cannot pass this check on Apify's proxy IPs.
Fix (reverted). Added a 4th, final proxy-ladder tier: a real headless
Chromium browser (Playwright). See 0.4 above for why this was rolled back.
[0.2] - 2026-08-29
Fixed - 0 results on every run
Root cause. Google retired the standalone full-page Events vertical this
actor relied on (GET /search?...&ibp=htl;events). That endpoint still
answers 200 OK with a real page shell, but its results panel always
rendered "Can't find events that match." โ verified live for both a small
market (events in South Bend Indiana) and a huge one (
concerts innewyork
), reproduced identically in a full, modern, JS-executing Chrome
session (not just via the plain-HTTP transport this actor uses). Because the
endpoint returned 200 with no qlMead markup and no known block marker,
extracto.looks_blocked() never flagged it as a problem either, so the
actor's direct โ datacenter โ residential proxy ladder never had a reason to
escalate โ every run quietly "succeeded" with zero events, page after page.
Fix.
Switched transport to the classic GET /search?q=... endpoint (no
ibp/start params). Google still publishes the same event data, now as
an inline "Events" carousel embedded in the organic search results page.
Rewrote extracto.py's selectors and parsing for the current markup
(div.NCvvle cards, icon-classified detail rows) and added
extract_deferred_fragments() to also recover the cards Google ships as
JS-escaped HTML strings for below-the-fold rendering (
jsl.dh(this.id,"...")
) โ without it, only the first 3-4 of ~10 events per query would be
found.
Fixed looks_blocked(): it now recognises Google's JS-required
interstitial (/httpservice/retry/enablejs) as a block, while correctly
NOT treating that marker's presence as a block when it's just the
<noscript> fallback boilerplate every real SERP also ships (checked via
the stable id="rso" / id="search" organic-results markers first).
Unrecognised page shapes now fail closed (treated as blocked, escalating
the proxy ladder) instead of failing open as a silent "no events".
Fixed build_query() folding the configured location into the query
text even when it was already present but for a punctuation difference
("South Bend Indiana" vs "South Bend, Indiana") โ the resulting
duplicated query ("...Indiana in South Bend, Indiana") reliably made
Google drop the Events carousel entirely.
maxPages is now capped to one effective page per query: Google's inline
Events carousel ships a single fixed batch on page 1 of results only โ
start=10/20/... returns further organic results with no Events section
at all, so there was nothing further to paginate into.
Known limitation (not a bug): the compact card format Google now uses
for this carousel no longer exposes an outbound ticket/event link, venue
star rating/review count, or a map thumbnail in the static HTML โ those are
now behind a JS-driven detail panel that opens on click. The corresponding
output fields (link, venue.rating, venue.reviews,
event_location_map.image) are kept for schema compatibility but are
honestly empty/null rather than guessed. Also, the Events carousel itself
does not appear for every market (e.g. it was not observed for New York
queries during testing, with or without a location hint) โ when Google
genuinely doesn't show it, the actor now correctly reports 0 events for that
query instead of erroring.
