Google Hotels Scraper - Multi-Query OTA Price Comparison
Pricing
from $22.50 / 1,000 hotel results
Search up to 5 hotel queries in one run and compare live prices across 30+ OTAs including Booking.com, Expedia and Agoda. Every hotel returns overview, OTA offers with booking links, rating, property type, reviews, photos and amenities. Stable per-stay keys make daily rate monitoring diffable.
All notable changes to this project should be documented in this file.
US$450 is no longer labelled SGD, and CA$120 no longer AUD. The card and
detail reads checked includes('S$') and includes('A$') with no left
boundary, so both matched inside the longer prefix. Every price path (card,
detail, entity panel, HTTP rescue) now reads currency with one shared,
left-anchored detectRenderedCurrency, which also names AU$, SG$,
MX$ and CN¥/RMB. ¥ and £ are anchored too, so CN¥500 is not yen
and E£ 120 is not sterling. It runs in Node on the text the browser hands
out. A bare $ on an OTA offer is the requested currency when that is a
dollar currency (USD, CAD, AUD, SGD, NZD, HKD, TWD, MXN) and USD otherwise,
so sg/SGD and au/AUD runs no longer trip the misquote stop on their own
prices. The entity panel's page-wide evidence scan reads ISO codes from the
22 input currencies only, so "TOP 10" or "TRY 2 nights" is not evidence.
A search Google priced in another currency is retried on a fresh exit instead
of finishing empty. Run 2n14qm51edMUwr7XF (us/USD, 1.8.26) carried a USD
ts in the address bar and on every card link while every card was labelled
SGD. Those labels were most likely misparsed US$ prices (see above); the
genuine misquote on record is run UOm6DPN1BOjD4SoJH, whose OTA links
carried cur=SCR. In 2n14q the token check passed, the request finished on
its first attempt (retryHistogram: [1]) and all 10 rows were rejected.
Cards whose first priced entries all name another currency now throw
QuotedCurrencyMismatchError before any detail page is opened, and so does
the detail loop's three-misquote stop when publishing would bill nothing and
the dataset holds no row of the query yet. Both throw before publishing, so
the retry cannot deliver a row twice. Retries stay at MAX_REQUEST_RETRIES,
then the HTTP rescue runs on a fresh session. A query still misquoted after
all of that fails the run with "Google quoted SCR instead of USD on every
exit tried…" in place of the advice to match a pair that was already matched.
When another query failed for a different reason, the message names the
misquoted queries instead of making the run-wide claim.
Prices quoted in another currency are no longer published, or charged, as
the requested one. Run UOm6DPN1BOjD4SoJH (US/USD) delivered Seychelles rupee
prices labelled USD, about 14x too high, as validated and charged 8 of them:
the exit IP geolocated to the Seychelles, Google quoted SCR 8,536, the
price-text read ignored ISO codes and fell back to the requested USD, and the
context gate compared the currency=USD URL parameter, which is Google's echo
of the request. The opaque ts token is what sets the currency (a USD token
rendered $ from a KR exit, currency=/curr= did not), so the search
refresh, every detail page and the entity re-navigation now ask with the
requested currency in the token; a search still misquoted after that throws
QuotedCurrencyMismatchError, which retires the exit before the retry. Rows
are rejected as price-context mismatches when the token's currency, an ISO
code in the rendered price (SCR 6,204), or a shopper-currency parameter on
any OTA booking link (cur, user_currency, mpg, Agoda's Currency, …)
names a currency other than the requested one. Booking links count only for
(merchant, key) pairs observed naming the real quote in two currencies, so an
unlisted merchant never rejects a row. The entity panel re-asks on a currency
withhold, the detail loop stops after three consecutive misquoted pages, and
an HTTP rescue after a misquoted exit runs on a fresh proxy session and drops
prices it cannot confirm. The same defect is in earlier runs:
YlUZjgDdSbfgpmwuN (us/USD, 20 SCR rows), v2sEV9ufL778L6Dft (us/USD, GBP)
and BHcXwDPGBz7LbuMpS (sg/USD, SGD).
Rows past the buyer's spending limit are no longer given away. Actor.charge()
does not refuse a charge over maxTotalChargeUsd — it resolves with a lower
chargedCount — and rows were pushed before the charge, so a $0.30 run on
1.8.24 delivered 20 rows against 12 charges. publishHotels now asks the
charging manager how many hotel-result events the budget still pays for
before pushing, pushes only that many billable rows (free rows unchanged),
charges exactly what it pushed, and runs that sequence one query at a time so
concurrent queries cannot read the same budget. Once the budget is spent,
queued queries, detail pages, retries and HTTP rescues stop, and the status
message leads with "Stopped at the spending limit (maxTotalChargeUsd): N rows
not delivered".
OTA offers carry their price again. The detail path handed the rendered price
text — which keeps its currency symbol, because the currency detection beside
it needs one — to parseLocalizedNumber, which reads a bare number and
rejects any group that is not all digits. Every offer therefore parsed as
null and was published as price: 0 beside a correct provider and a working
booking link. It went unseen because the entity path, which uses
parsePriceAmount and is correct, was the one answering while the card
selector was blind; build 1.8.18 taught the selector Google's second layout
and routed runs back through the broken parse. Five runs on 1.8.18 and 1.8.20
delivered 631 zero-priced offers, all of them charged.
Suffix-marker and unmarked locales keep their offer prices. Reading the
amount only where a currency marker sits beside it left Thailand, the
Philippines, Vietnam, Indonesia, Malaysia and suffix-rendered euro prices at
0, because the in-page trim keeps prefix markers only and hands those
locales bare digits. With the charging guard below, a zero is no longer just a
wrong number — it withholds the row, so those runs would have delivered every
hotel for free. The marker-anchored read is still tried first; the digits of
the already-trimmed token are read only when it declines.
An apostrophe no longer poses as a currency marker. parsePriceAmount read
"1\u2019234" as 1, matching the digits before the apostrophe and treating the
apostrophe as the marker beside them — and because a marker match is
authoritative, the caller never reached the digit run that reads 1234. Both
apostrophes join . and , as grouping marks the marker class refuses.
A price grouped with a typographic apostrophe or a thin space is read whole. Neither separator was in the digit run, so "$1\u2019234" was truncated to "$1" and published as a price of 1 — small, plausible, and charged. Both the shared parser and the two in-page trims now group on them.
A hotel whose offers carry no price is no longer billed.
classifyOtaPriceStatus counted offers, so a set of 34 zero-priced offers
looked as complete as a set of 34 real ones. It now requires one offer with a
positive price; a set without one is missing, which delivers the row and
withholds the charge, exactly as an empty set already did.
A hotel's own name stays out of its amenities. The badge row is split on
commas and a hotel may have commas in its name, so
"Staypineapple, An Elegant Hotel, Union Square San Francisco" published
"An Elegant Hotel" and "Union Square San Francisco" as facilities. Fragments
matching a whole segment of the property's name are dropped; a substring match
is not enough, so "Parking" survives at the "Parking Garden Hotel".
Multi-query runs retry their requests again. maxRequestRetries was
queries.length > 1 ? 0 : 1, so every query in a 2–5 query run got exactly one
attempt: one bad exit IP took the query down, and a run whose queries all lost
that single attempt is failed outright by summary.ts. The trade it was making
never existed — Crawlee re-enqueues a retry instead of running it inline, so a
retried query never blocked the queries behind it. Every run shape now uses
MAX_REQUEST_RETRIES (1).
A batch is never delivered or charged twice by a retry. publishHotels on the
card list path was unguarded, and it reports a charge failure in its result —
so an escaping throw is pushData, with an unknown part of the batch already
written. That failed the request, and the retry re-scraped and re-published the
whole batch; deduplication is per batch, so the rows that had landed were
pushed and charged a second time. With scrapeOtaPrices: false nothing even
slowed the second pass down, because the already-scraped skip runs on the
detail path only. The query is now recorded as failed in place, matching the
guard the entity path already had.
A retry the Actor deadline cannot fund is cancelled instead of started. The
per-request timeout is derived by dividing usable runtime by concurrency waves
counted from the query list, which budgets one attempt per request, so retries
spend time nothing reserved — five queries at maxResults: 100 reached 7440 s
of worst-case work against a 3600 s deadline. Overrunning it kills the
container mid-request: crawler.run() never returns, no SUMMARY is written
and no status message is set, while the rows already pushed stay charged. The
errorHandler now sets request.noRetry when the remaining runtime cannot
cover another navigation-plus-handler attempt, and the check runs again before
each retry, so no attempt is authorized unless the deadline could fund it in
full at that moment; a cancelled retry falls through to the HTTP rescue. The
guarantee is per authorized attempt, not a wall-clock bound on the run — an
authorized retry is re-enqueued behind whatever is already queued, so it can
still start later than the moment it was approved. Default single-query runs
are unaffected.
A navigation timeout now retires the proxy session before the retry. Crawlee
retires by itself only for its ROTATE_PROXY_ERRORS list; a navigation timeout
goes through handleRequestTimeout(), which just calls session.markBad() —
one point against a default maxErrorScore of 3 — so the retry went back out
over the exit IP that had just timed out. This is the trap already recorded for
the consent wall, on the one failure path the router never sees. The crawler's
new errorHandler retires when isExitIpFailure() claims the error
(navigation timeouts in either Crawlee's or Playwright's wording, net::ERR_*
transport failures) and leaves the session alone otherwise, so an input or
selector failure does not throw away a working residential session. The
stay-context refresh in routes.ts navigates from inside the request handler,
where Crawlee never sees the failure, so its rethrow now carries the original
message instead of replacing it with "refresh failed".
Priced rows recovered by the HTTP rescue are delivered again. The rescue
stamped rows with its own fetch URL, which carries q/hl/gl/currency
but not checkin/checkout/adults, so the returned-context gate in
publishHotels rejected every priced rescue row ("missing visible context")
and the rescue reported "delivered no rows" over data it had already
recovered. Measured 2026-08-25 on 1.8.4: "Marina Bay Sands Singapore"
(sg/SGD) failed in 282 s with the rescue's one priced row discarded. Rows now
carry the canonical search URL built from the exact context the rescue rpc
requested, which the gate validates. Browser-path rows are unaffected.
Rescued rows now pass through the same delivery contracts as browser rows:
the rating/price filters and, for a quoted query, the relevance guard. The
rescue paths skipped them — invisibly, while the context gate was discarding
every priced rescue row anyway. Without this, a "quoted hotel" query
recovered over HTTP would charge for area alternatives the browser path
rejects.
![Expedia.com & Hotel.com [$2.5💰] Reviews/Ratings/Sentiment A. avatar](https://images.apifyusercontent.com/S3jQnQO1Qq7Evmm8G6HhWSwePurMx4mbNBaxjT2ShAg/rs:fill:76:76/cb:1/aHR0cHM6Ly9hcGlmeS1pbWFnZS11cGxvYWRzLXByb2QuczMudXMtZWFzdC0xLmFtYXpvbmF3cy5jb20vbmduOHZSRVNUa3NleVdFNnAtYWN0b3ItNkkxaEJLTk1nQk5IZVl4dUwtOXNoSm5hbnd2Ni1leC5wbmc.webp){kind=avatar}
proxy-groups.ts rather than written out, so advice cannot outlive
the pool. The per-query summary reason names the requested pair too.Overview, OTA Prices). The dataset schema previously
declared fields but no views, so results rendered as raw JSON instead of a
table — and Apify refuses to publish a task for an Actor with no view, which
blocked the Store landing page entirely.Hotel detail scraping navigates to each hotel's own detail URL instead of
clicking the card at a given index and navigating back. The click-and-return
approach coupled every hotel to the results list surviving intact, so a single
failed click or back-navigation stranded the page and every later hotel logged
Card index N out of range (0 links). Google also recycles cards out of the
DOM while scrolling, so the click target went missing on healthy runs too.
Hotel detail scraping no longer stops after the first hotel. When the run
could not get back to the search results list, every later hotel resolved
against an empty list and logged Card index N out of range (0 links), so a
20-hotel query attempted exactly one detail page. Recovery now retries the
saved search URL with a real navigation wait and dismisses a re-shown consent
dialog; when recovery genuinely fails the detail loop stops and records a
partial reason instead of burning the remaining runtime.
OTA offer rows are given a 15s hydration budget instead of 3s. Google Hotels
renders the price panel well after domcontentloaded, and residential proxies
add several seconds more, so the old budget expired before any offer appeared.
OTA offers are no longer discarded because their booking link is a Google
redirect. Google wraps every offer's deep link, so the terminal-URL rule
rejected 100% of offers — throwing away the provider and price alongside the
link, which is the price comparison this Actor exists to deliver. Offers are
kept and the link is labelled bookingUrlStatus: "google_redirect". Embedded
redirect targets are still never unwrapped, and malformed, credentialed, or
non-HTTP links are still rejected.
Deduplication no longer collapses an entire batch into one row when Google
serves entity-form URLs. Those URLs carry the hotel identity in the path
(/entity/{id}/prices) and set the same qs=OAA stub on every card, so
reading qs gave all 20 hotels one identity and 19 were dropped before output
and charging. Identity now reads the entity id first and ignores qs stubs
shorter than a real token. This hit whole markets: the hotels in Seoul
smoke run delivered 1 of 20 hotels before the fix.
Search cards fall back to the hotel name link when the "View prices" anchor is missing. That URL is both the detail-scrape target and the row's dedupe identity, so an absent anchor previously gave every card in the batch the same empty identity and collapsed 20 results into 1.
A batch whose rows mostly collapse during deduplication now logs a warning with sample URLs instead of only incrementing a counter in the run summary.
otaPriceStatus: "missing", and
the run logs how many were withheld. Previously a broken detail scrape billed
the buyer in full for the OTA price comparison it silently dropped.otaPriceStatus output field (complete | missing) on every row.bookingUrlStatus field (direct | google_redirect) on every OTA offer.withheldForMissingOtaPrices in the publish result and run summary.