# Changelog of Google Hotels Scraper - Multi-Query OTA Price Comparison (`jy-labs/google-hotels-multi-query-scraper`) Actor - **URL**: https://apify.com/jy-labs/google-hotels-multi-query-scraper/changelog.md - **Full Actor documentation**: https://apify.com/jy-labs/google-hotels-multi-query-scraper.md ## Changelog All notable changes to this project should be documented in this file. ### \[Unreleased] #### Fixed - `US$450` is no longer labelled SGD, and `CA$120` no longer AUD. The card and detail reads checked `includes('S$')` and `includes('A$')` with no left boundary, so both matched inside the longer prefix. Every price path (card, detail, entity panel, HTTP rescue) now reads currency with one shared, left-anchored `detectRenderedCurrency`, which also names `AU$`, `SG$`, `MX$` and `CN¥`/`RMB`. `¥` and `£` are anchored too, so `CN¥500` is not yen and `E£ 120` is not sterling. It runs in Node on the text the browser hands out. A bare `$` on an OTA offer is the requested currency when that is a dollar currency (USD, CAD, AUD, SGD, NZD, HKD, TWD, MXN) and USD otherwise, so sg/SGD and au/AUD runs no longer trip the misquote stop on their own prices. The entity panel's page-wide evidence scan reads ISO codes from the 22 input currencies only, so "TOP 10" or "TRY 2 nights" is not evidence. - A search Google priced in another currency is retried on a fresh exit instead of finishing empty. Run `2n14qm51edMUwr7XF` (us/USD, 1.8.26) carried a USD `ts` in the address bar and on every card link while every card was labelled `SGD`. Those labels were most likely misparsed `US$` prices (see above); the genuine misquote on record is run `UOm6DPN1BOjD4SoJH`, whose OTA links carried `cur=SCR`. In 2n14q the token check passed, the request finished on its first attempt (`retryHistogram: [1]`) and all 10 rows were rejected. Cards whose first priced entries all name another currency now throw `QuotedCurrencyMismatchError` before any detail page is opened, and so does the detail loop's three-misquote stop when publishing would bill nothing and the dataset holds no row of the query yet. Both throw before publishing, so the retry cannot deliver a row twice. Retries stay at `MAX_REQUEST_RETRIES`, then the HTTP rescue runs on a fresh session. A query still misquoted after all of that fails the run with "Google quoted SCR instead of USD on every exit tried…" in place of the advice to match a pair that was already matched. When another query failed for a different reason, the message names the misquoted queries instead of making the run-wide claim. - Prices quoted in another currency are no longer published, or charged, as the requested one. Run `UOm6DPN1BOjD4SoJH` (US/USD) delivered Seychelles rupee prices labelled USD, about 14x too high, as `validated` and charged 8 of them: the exit IP geolocated to the Seychelles, Google quoted `SCR 8,536`, the price-text read ignored ISO codes and fell back to the requested USD, and the context gate compared the `currency=USD` URL parameter, which is Google's echo of the request. The opaque `ts` token is what sets the currency (a USD token rendered `$` from a KR exit, `currency=`/`curr=` did not), so the search refresh, every detail page and the entity re-navigation now ask with the requested currency in the token; a search still misquoted after that throws `QuotedCurrencyMismatchError`, which retires the exit before the retry. Rows are rejected as price-context mismatches when the token's currency, an ISO code in the rendered price (`SCR 6,204`), or a shopper-currency parameter on any OTA booking link (`cur`, `user_currency`, `mpg`, Agoda's `Currency`, …) names a currency other than the requested one. Booking links count only for (merchant, key) pairs observed naming the real quote in two currencies, so an unlisted merchant never rejects a row. The entity panel re-asks on a currency withhold, the detail loop stops after three consecutive misquoted pages, and an HTTP rescue after a misquoted exit runs on a fresh proxy session and drops prices it cannot confirm. The same defect is in earlier runs: `YlUZjgDdSbfgpmwuN` (us/USD, 20 SCR rows), `v2sEV9ufL778L6Dft` (us/USD, GBP) and `BHcXwDPGBz7LbuMpS` (sg/USD, SGD). - Rows past the buyer's spending limit are no longer given away. `Actor.charge()` does not refuse a charge over `maxTotalChargeUsd` — it resolves with a lower `chargedCount` — and rows were pushed before the charge, so a $0.30 run on 1.8.24 delivered 20 rows against 12 charges. `publishHotels` now asks the charging manager how many `hotel-result` events the budget still pays for before pushing, pushes only that many billable rows (free rows unchanged), charges exactly what it pushed, and runs that sequence one query at a time so concurrent queries cannot read the same budget. Once the budget is spent, queued queries, detail pages, retries and HTTP rescues stop, and the status message leads with "Stopped at the spending limit (maxTotalChargeUsd): N rows not delivered". - OTA offers carry their price again. The detail path handed the rendered price text — which keeps its currency symbol, because the currency detection beside it needs one — to `parseLocalizedNumber`, which reads a bare number and rejects any group that is not all digits. Every offer therefore parsed as `null` and was published as `price: 0` beside a correct provider and a working booking link. It went unseen because the entity path, which uses `parsePriceAmount` and is correct, was the one answering while the card selector was blind; build 1.8.18 taught the selector Google's second layout and routed runs back through the broken parse. Five runs on 1.8.18 and 1.8.20 delivered 631 zero-priced offers, all of them charged. - Suffix-marker and unmarked locales keep their offer prices. Reading the amount only where a currency marker sits beside it left Thailand, the Philippines, Vietnam, Indonesia, Malaysia and suffix-rendered euro prices at `0`, because the in-page trim keeps prefix markers only and hands those locales bare digits. With the charging guard below, a zero is no longer just a wrong number — it withholds the row, so those runs would have delivered every hotel for free. The marker-anchored read is still tried first; the digits of the already-trimmed token are read only when it declines. - An apostrophe no longer poses as a currency marker. `parsePriceAmount` read "1\u2019234" as 1, matching the digits before the apostrophe and treating the apostrophe as the marker beside them — and because a marker match is authoritative, the caller never reached the digit run that reads 1234. Both apostrophes join `.` and `,` as grouping marks the marker class refuses. - A price grouped with a typographic apostrophe or a thin space is read whole. Neither separator was in the digit run, so "$1\u2019234" was truncated to "$1" and published as a price of 1 — small, plausible, and charged. Both the shared parser and the two in-page trims now group on them. - A hotel whose offers carry no price is no longer billed. `classifyOtaPriceStatus` counted offers, so a set of 34 zero-priced offers looked as complete as a set of 34 real ones. It now requires one offer with a positive price; a set without one is `missing`, which delivers the row and withholds the charge, exactly as an empty set already did. - A hotel's own name stays out of its `amenities`. The badge row is split on commas and a hotel may have commas in its name, so "Staypineapple, An Elegant Hotel, Union Square San Francisco" published "An Elegant Hotel" and "Union Square San Francisco" as facilities. Fragments matching a whole segment of the property's name are dropped; a substring match is not enough, so "Parking" survives at the "Parking Garden Hotel". - Multi-query runs retry their requests again. `maxRequestRetries` was `queries.length > 1 ? 0 : 1`, so every query in a 2–5 query run got exactly one attempt: one bad exit IP took the query down, and a run whose queries all lost that single attempt is failed outright by `summary.ts`. The trade it was making never existed — Crawlee re-enqueues a retry instead of running it inline, so a retried query never blocked the queries behind it. Every run shape now uses `MAX_REQUEST_RETRIES` (1). - A batch is never delivered or charged twice by a retry. `publishHotels` on the card list path was unguarded, and it reports a charge failure in its result — so an escaping throw is `pushData`, with an unknown part of the batch already written. That failed the request, and the retry re-scraped and re-published the whole batch; deduplication is per batch, so the rows that had landed were pushed and charged a second time. With `scrapeOtaPrices: false` nothing even slowed the second pass down, because the already-scraped skip runs on the detail path only. The query is now recorded as failed in place, matching the guard the entity path already had. - A retry the Actor deadline cannot fund is cancelled instead of started. The per-request timeout is derived by dividing usable runtime by concurrency waves counted from the query list, which budgets one attempt per request, so retries spend time nothing reserved — five queries at `maxResults: 100` reached 7440 s of worst-case work against a 3600 s deadline. Overrunning it kills the container mid-request: `crawler.run()` never returns, no `SUMMARY` is written and no status message is set, while the rows already pushed stay charged. The `errorHandler` now sets `request.noRetry` when the remaining runtime cannot cover another navigation-plus-handler attempt, and the check runs again before each retry, so no attempt is authorized unless the deadline could fund it in full at that moment; a cancelled retry falls through to the HTTP rescue. The guarantee is per authorized attempt, not a wall-clock bound on the run — an authorized retry is re-enqueued behind whatever is already queued, so it can still start later than the moment it was approved. Default single-query runs are unaffected. - A navigation timeout now retires the proxy session before the retry. Crawlee retires by itself only for its `ROTATE_PROXY_ERRORS` list; a navigation timeout goes through `handleRequestTimeout()`, which just calls `session.markBad()` — one point against a default `maxErrorScore` of 3 — so the retry went back out over the exit IP that had just timed out. This is the trap already recorded for the consent wall, on the one failure path the router never sees. The crawler's new `errorHandler` retires when `isExitIpFailure()` claims the error (navigation timeouts in either Crawlee's or Playwright's wording, `net::ERR_*` transport failures) and leaves the session alone otherwise, so an input or selector failure does not throw away a working residential session. The stay-context refresh in `routes.ts` navigates from inside the request handler, where Crawlee never sees the failure, so its rethrow now carries the original message instead of replacing it with "refresh failed". - Priced rows recovered by the HTTP rescue are delivered again. The rescue stamped rows with its own fetch URL, which carries `q`/`hl`/`gl`/`currency` but not `checkin`/`checkout`/`adults`, so the returned-context gate in `publishHotels` rejected every priced rescue row ("missing visible context") and the rescue reported "delivered no rows" over data it had already recovered. Measured 2026-08-25 on 1.8.4: "Marina Bay Sands Singapore" (sg/SGD) failed in 282 s with the rescue's one priced row discarded. Rows now carry the canonical search URL built from the exact context the rescue rpc requested, which the gate validates. Browser-path rows are unaffected. - Rescued rows now pass through the same delivery contracts as browser rows: the rating/price filters and, for a quoted query, the relevance guard. The rescue paths skipped them — invisibly, while the context gate was discarding every priced rescue row anyway. Without this, a `"quoted hotel"` query recovered over HTTP would charge for area alternatives the browser path rejects. #### Changed - A run that the price-context gate blocked says why. Nothing pushed with rows rejected still fails the run — the gate is fail-closed on purpose — but the status message no longer reads "Done (failed). Pushed 0", which looks like a broken scraper. It now names the count of withheld rows, the requested search context, the country/currency pair as its usual cause (Google quotes in the exit IP's currency and ignores the currency parameter), and the fix. It blames the context rather than the currency alone because the gate also rejects rows on stay dates, guests and query, and the datacenter-served pair it advises is read from `proxy-groups.ts` rather than written out, so advice cannot outlive the pool. The per-query summary reason names the requested pair too. #### Added - Dataset views (`Overview`, `OTA Prices`). The dataset schema previously declared fields but no views, so results rendered as raw JSON instead of a table — and Apify refuses to publish a task for an Actor with no view, which blocked the Store landing page entirely. #### Fixed - Hotel detail scraping navigates to each hotel's own detail URL instead of clicking the card at a given index and navigating back. The click-and-return approach coupled every hotel to the results list surviving intact, so a single failed click or back-navigation stranded the page and every later hotel logged `Card index N out of range (0 links)`. Google also recycles cards out of the DOM while scrolling, so the click target went missing on healthy runs too. - Hotel detail scraping no longer stops after the first hotel. When the run could not get back to the search results list, every later hotel resolved against an empty list and logged `Card index N out of range (0 links)`, so a 20-hotel query attempted exactly one detail page. Recovery now retries the saved search URL with a real navigation wait and dismisses a re-shown consent dialog; when recovery genuinely fails the detail loop stops and records a partial reason instead of burning the remaining runtime. - OTA offer rows are given a 15s hydration budget instead of 3s. Google Hotels renders the price panel well after `domcontentloaded`, and residential proxies add several seconds more, so the old budget expired before any offer appeared. - OTA offers are no longer discarded because their booking link is a Google redirect. Google wraps every offer's deep link, so the terminal-URL rule rejected 100% of offers — throwing away the provider and price alongside the link, which is the price comparison this Actor exists to deliver. Offers are kept and the link is labelled `bookingUrlStatus: "google_redirect"`. Embedded redirect targets are still never unwrapped, and malformed, credentialed, or non-HTTP links are still rejected. - Deduplication no longer collapses an entire batch into one row when Google serves entity-form URLs. Those URLs carry the hotel identity in the path (`/entity/{id}/prices`) and set the same `qs=OAA` stub on every card, so reading `qs` gave all 20 hotels one identity and 19 were dropped before output and charging. Identity now reads the entity id first and ignores `qs` stubs shorter than a real token. This hit whole markets: the `hotels in Seoul` smoke run delivered 1 of 20 hotels before the fix. - Search cards fall back to the hotel name link when the "View prices" anchor is missing. That URL is both the detail-scrape target and the row's dedupe identity, so an absent anchor previously gave every card in the batch the same empty identity and collapsed 20 results into 1. - A batch whose rows mostly collapse during deduplication now logs a warning with sample URLs instead of only incrementing a counter in the run summary. #### Changed - Rows are no longer charged when detail scraping was requested but produced no OTA offers. They are still delivered, marked `otaPriceStatus: "missing"`, and the run logs how many were withheld. Previously a broken detail scrape billed the buyer in full for the OTA price comparison it silently dropped. #### Added - `otaPriceStatus` output field (`complete` | `missing`) on every row. - `bookingUrlStatus` field (`direct` | `google_redirect`) on every OTA offer. - `withheldForMissingOtaPrices` in the publish result and run summary.