# Changelog of Google Hotels Scraper & Price Tracker (`myth84/google-hotels-scraper`) Actor - **URL**: https://apify.com/myth84/google-hotels-scraper/changelog.md - **Full Actor documentation**: https://apify.com/myth84/google-hotels-scraper.md ### Changelog History of `google-hotels-scraper` (folder `actors/google-hotels`). Versions are the Apify build numbers. #### Unreleased (free cancellation, `freeCancellationOnly` and the counters of each search) - Brief `docs/cloud-tasks/2026-10-23_google-hotels-extend-cancellation-counts.md` (2026-10-23). No schema change: the row stays `travel-listing/1.7`, the deadline goes into the existing `cancellationPolicy` with the studio's convention (`booking-hotel-details`), so no new field and no second minor version after the all-in price. - **`cancellationPolicy`** is filled from the prices answer (`lvkpBc`), hence with `includeOffers`, `hotelUrls` or `freeCancellationOnly`: Google gives cancellation terms only for the room rates of the sponsored booking-site blocks (`h[6][16][0]`, `h[6][17][0]`, rates `e[24][*]`); the organic offers of `offers` carry none. A rate is free to cancel when it has feature code 19 or an English "Free cancellation until " label: in the three prices captures (314 rates) the code and the label go together on every rate, code 2 being breakfast. Values: `free_cancellation_until:` with the earliest deadline of those rates (every free-cancellation rate shown can be cancelled at least until that day; the year is the last one that puts the day on or before the check-in, never before the scrape day), `free_cancellation` when no label gives a readable deadline (another language than English), `null` otherwise. Captures: Brown's Avenue `free_cancellation_until:2026-11-17` (Priceline's rate, the other sites the 18th), The Central House 2026-11-18 in USD and 2026-11-17 in EUR (Hostelworld). The README example shows it. - **`freeCancellationOnly`** (boolean, default `false`): keeps only the hotels with a free-cancellation rate. Applied by the Actor (no Google filter was found in the audit), so it turns `includeOffers` on; a hotel left out is never charged, the next new hotels of the same page take its place before the next page is requested; a hotel whose rates cannot be fetched is left out too (its terms are unknown). Same rule for `hotelUrls`. A later search of the run counts the hotels an earlier one left out without asking Google again. Part of the price-monitoring identity only when on (the existing records keep their keys). - **Counters of each search**, for "I asked for N hotels and got fewer": `RUN_STATS.searches` now gives `kind`, `requested`, `announced` (Google's total, 15,000 at most), `served` (distinct hotels of the pages Google served; for `hotelUrls`, the hotels whose rates were obtained), `returned`, `pages`, `notReturned` (`beyondMaxResults`, `vacationRentals`, `foundEarlier`, `noFreeCancellation`, `unchanged`, `errors`, `chargeLimit`, `runStopped`), `stopReason` (`max_results`, `last_page`, `exhausted`, `page_cap`, `request_cap`, `charge_limit`, `page_failed`, `all_hotels`, `not_searched`) and `explanation` (one sentence, also the last log line of the search); the first fields (`search`, `hotels`, `duplicatesSkipped`, `announced`, `searched`) are unchanged. `served` = `returned` + the sum of `notReturned` (checked by every crawler test). The same list goes to a new **`SUMMARY`** record of the default key-value store (`google-hotels-summary/1.0`, in the output schema), and, shortened, to the terminal status message ("60 hotels · 9 requests · 0 blocked · hotels in Lisbon 60/60 (maxResults), 15,000 announced, 134 served", 250 characters at most, "+N more in SUMMARY" beyond): the shared runner writes that message, `main.ts` wraps `Actor.setStatusMessage` to append the counters to the terminal message only. - New counters `extra.freeCancellation` (hotels written with a free-cancellation rate) and `extra.noFreeCancellation` in `RUN_STATS` and the "Done" log line. The message of a run without any result mentions `SUMMARY` for `freeCancellationOnly`. - README: `cancellationPolicy` bullet, input row, *Free cancellation* in *Booking sites and direct links*, a *Limits* line on the counters, *What is not included*, AI agents, alternatives and pricing table; dataset schema: description of `cancellationPolicy`, shown in the *Booking sites* view. - Not verified offline (first run on the Mac): the share of hotels whose prices answer has sponsored rates (2 of 2 in the captures), labels and deadlines in other languages (`hl` other than `en`), the share of hotels kept by the filter and the extra rate calls it costs, the status message as the Console shows it. - Not changed: prices, example tasks, Store texts. #### Unreleased (all-in price: `taxesIncluded` and `priceTotalAllIn`) - Brief `docs/cloud-tasks/2026-10-23_travel-prices-all-in.md` (2026-10-23): rows are **`travel-listing/1.7`** (the price-comparison schema, formerly 1.4, on top of the search schema `travel-listing/1.5`), two columns after `taxesAndFees`, nothing else changes (`priceTotal` keeps its meaning: the lowest stay price with taxes and fees). - **`taxesIncluded`** = `true` whenever there is a `priceTotal` (the last item of Google's `[base, taxes, fees, total]` of the stay; a test checks on every priced hotel of the committed searches that the total is the sum of the three), `null` without a price. - **`priceTotalAllIn`** = `priceTotal`, `null` without a price (`allInPriceTotal` of `@scraper-studio/core/schema`). The offers of `offers` are unchanged (they already carry their prices before and with taxes). - Dataset schema: both columns documented, `priceTotal` no longer called "the column to compare" (that is `priceTotalAllIn`); *Overview* shows `priceTotalAllIn` next to `priceTotal`, *Prices* shows both after `priceTotal`. README: `priceTotal` / `priceTotalAllIn` bullet, FAQ on taxes, sample output with `priceTotalAllIn` (instead of `propertyType`), the strengths worth knowing when switching (hard `maxResults` cap per search never charged beyond, `onlyChanges`), *More scrapers* pointing at `priceTotalAllIn`. #### Unreleased (repair: a call Google refuses no longer fails the whole run) - Brief `docs/cloud-tasks/2026-10-22_google-hotels-fix-release.md` (2026-10-22), PR #77. The control run of the release after PR #76 (`Yc79YVYWEcLxCjnLm`, `inputs/control-scale.json`: Paris and Rome, 300 hotels each, without the booking-site rates) ended as FAILED after 467 hotels, without `RUN_STATS`, on the log line `ERROR Run failed`. The run log was not available to the repair session; read from the code, the only exception of the crawler reachable in that run (no monitoring, no booking-site rates) once rows were written is `GoogleRejectedError`: one `AtySUc` call answered by `wrb.fr` with an error code, which 0.1 never retried and turned into a failure of the whole run (a mid-run "format change", although the previous pages of the same request had been served). - **A `wrb.fr` answer with an error code is now an unusable answer like the others**: retried 3 times on a new proxy session with a pause of 1, 2, 4 s, then the page is abandoned (the search stops with its hotels, the next searches run) or the hotel is written without its booking-site rates (`offersIncluded: false`), or, for `hotelUrls`, abandoned. The rows already written are kept, `RUN_STATS` is written and the run succeeds. - The run fails with `GoogleRejectedError` ("Google rejected the request (code N): the request format changed") only when it obtained no hotel and the last failure was a refused call: a real change of the request format still gives the explicit message, after the retries. - New counter `extra.rejected` (attempts refused by Google) in `RUN_STATS` and in the "Done" log line; the reason of each refused attempt is logged with its code ("Google refused the call, error code N"), so that the next control says which code Google sent. - Price monitoring stays fail-closed: a search with an abandoned page is an error, its record is not updated. - Tests: a page refused once then served (no error, 1 retry); every attempt of page 2 refused (rows of page 1 kept, next search run, 1 error); a refused prices call (hotel written without rates); every call refused with nothing obtained (explicit error after 1 + 3 attempts). The four fail on 0.1. The committed inputs are also checked against `requestBudgetWarning` (`maxRequests` enough for `maxResults` per search). - Not changed: inputs, Store texts, example tasks, output schema, prices. #### 0.1 — 2026-10-22 - First version, written offline on the committed fixtures of the audit (`docs/audits/google-hotels.md`, brief `docs/cloud-tasks/2026-10-22_google-hotels-build.md`). Source: the `batchexecute` RPC endpoint of the Google Travel web app, `AtySUc` for the hotel search (20 hotels per call, 18 in a variant) and `lvkpBc` for the rate of every booking site of one hotel. The only URL ever requested is `https://www.google.com/_/TravelFrontendUi/data/batchexecute` (checked before every request, and by a test over every request of the crawler tests): never `/travel/search`, `/travel/entity`, `/travel/clk`, `/travel/lodging/clk` nor `/aclk`, disallowed by robots.txt. Requests are byte for byte those of the captures (URL with the `bl` build label of the audit, headers without cookie, `f.req` form body); `Accept-Language` follows `hl`. - Transport `got` (the default of `httpRequest`), Apify **datacenter** proxies by default, a **new proxy session per attempt**, no cookie. An unusable answer (status ≠ 200, HTML page, body without `wrb.fr`) is retried 3 times with a pause of 1, 2, 4 s, then the page (or the hotel) is abandoned with a log line; a `wrb.fr` answer with an error code fails the run at once ("Google rejected the request (code 3)"). Usability is decided by parsing the answer, not by the generic block markers, which could match a hotel description ("just a moment from the beach"). - Pagination by the token of each answer (`qs[1]`), global deduplication by entity id per stay (Google's order is not deterministic: about 16 new hotels per call), stops at `maxResults` hotels per search, without a next token, after two pages without a new hotel, or after `ceil(maxResults / 12) + 3` calls (30 at most). Hotels cut by a search's cap stay available to the next search of the run. The pages of a search run one after the other; the booking-site calls (`includeOffers`, `hotelUrls`) run `concurrency` at a time (default 3). - Inputs: `searchQueries` (prefilled with "hotels in Lisbon", no schema default so that runs made of `hotelUrls` do not search Lisbon too; `{}` still searches it), `startUrls` (Google Hotels search URLs read locally: `q`, the `ts` protobuf for dates, travellers and currency, `hl`, `gl`; other parameters ignored and listed in the log), `hotelUrls` (URLs with `/entity/` or ids, up to 500, split into tasks of `maxResults` so that the charge cap covers them), fixed or relative dates, adults (1–6), children's ages, `currency` (any ISO code; USD and EUR validated), `language` (`hl`), `country` (`gl`), `maxResults` (1–300 per search), `includeVacationRentals`, `includeOffers`, `maxOffersPerHotel`, price monitoring, `maxRequests`, `concurrency`, `delayMs`, `debugStoreResponses` (alias of the runner's `debugStoreHtml`). - Output schema `travel-listing/1.4` (`packages/core/src/schema/travel-prices.ts`, additive): every `travel-listing/1.2` column, then `pricePerNightWithTaxes`, `priceTotalBeforeTaxes`, `description`, `checkInTime`, `checkOutTime`, `website`, `ratingBreakdown`, `googleHotelId`, `googleMapsCid`, `googlePlaceId`, `offersIncluded`, `offers` (site, site id, official site, prices per night and of the stay before and with taxes, direct link decoded from `pcurl`), `offerCount`, `cheapestOfferSite`, `cheapestOfferPriceTotal`, `officialSitePriceTotal`. `pricePerNight` is Google's nightly price before taxes (base + fees per night), `priceTotal` the stay with taxes and fees, `taxesAndFees` = taxes + fees, `priceTotalBeforeTaxes` = base + fees (Google's "before taxes" total). Offers are sorted by stay price with taxes, offers without it last. - Local decisions on the fixtures: `country` is the English name of `countryCode`; `city` is the city Google resolved for the search (the hotel's own city for `hotelUrls`); vacation rentals keep `maxGuests`, `bedrooms` and `bathrooms` from `h[19][1]` but `beds` stays `null` (the 5th value of that array differs from the "N beds" Google displays for 11 of the 13 rentals of the captures, so it is not the number of beds); non-breaking spaces of Google's labels ("3:00 PM", "Hôtel 3 étoiles") become plain spaces; amenities of hotels come from the code table of the audit (§ 5 bis, codes without a label ignored), rental amenities are Google's own labels; in prices-only mode (`hotelUrls`) `amenities` is `[]` (the prices answer uses other codes). The address of the prices answer is not output (no column in the schema). - Price monitoring (model: `expedia-hotels`): `monitorPrices`, `monitorStoreName` (default `google-hotels-price-monitor`), `onlyChanges`; one record per search (`google-hotels-monitor/1`): search text or list of hotels, stay (fixed or relative), adults, children, currency, `hl`, `gl`, rentals included or not; compared price `priceTotal` (else `pricePerNight`); records written only after a clean crawl. - Not verified at build time (first run on the Mac): the occupancy of the prices RPC beyond 2 adults (children passed as the search echoes them), currencies other than USD and EUR, the real pagination depth, the pace at concurrency 3, the share of 18-hotel answers, the cost with offers. - Store readiness: README (Booking sites and direct links, Search from Google Hotels URLs, Price monitoring, What is not included, FAQ), input schema, dataset schema with Overview / Prices / Booking sites / Location / Changes views, output schema, 8 example tasks with relative dates on datacenter proxies, consistency tests (README ⇔ schemas ⇔ code, Ajv replay of the 121 fixture rows), crawler tests on a fake HTTP client.