Tripadvisor Reviews Scraper

This changelog summarizes all changes of the TripAdvisor Actors provided by Maximillian Copelli. The specific Actors that are affected are listed for each change.

2024-10-24

Features (tripadvisor)

• Added support for keyword search queries such as elephant or paddleboard
  • You can specify either location or keyword query using the new query input field

Fixes (tripadvisor)

• Fixed hotel search crawling and result extraction

Deprecations (tripadvisor)

• Deprecated locationFullName, use query instead (the Actor now supports both location and keyword queries)

2024-10-21

Fixes (tripadvisor)

• Fixed top-category restaurant search crawling from tourism page locationFullName input

2024-09-05

Features (tripadvisor)

• Added output field input to make it easier to match results with the requested scrape URLs

2024-08-30

Fixes (tripadvisor-places)

• Fixed extraction of listing images to fetch more results
  • All types of listings affected: hotels, restaurants, attractions, vacation rentals

2024-08-18

Features (tripadvisor-places)

• Added extraction of online order providers for restaurant listings
  • New output field orderOnline which is an array of objects with the following properties:
    • providerType, canProvideTimeslots, buttonText, offerURL, logoUrl, provider, providerId, providerDisplayName, headerText

2024-08-14

Features (tripadvisor-places)

• Added extraction of isClaimed info for restaurant listings. New output fields:
  • isClaimedIcon - is true when the isClaimed icon with a green checkmark was discovered on the page
  • isClaimedText - localized text such as Claimed / Unclaimed

2024-07-16

Features (tripadvisor-places)

• Added support for new format of public trips
  • new format: https://www.tripadvisor.com/TripDetails-t122547544
  • old format: https://www.tripadvisor.com/Trips/122547544

2024-07-14

Fixes (All TripAdvisor Actors)

• Fixed crawling issues that was causing the scraper to crash often

2024-07-01

Fixes (tripadvisor-places)

• Optimized requests enqueueing (total number of processed requests decreased)

2024-06-24

Features (tripadvisor-places)

• Added extraction of nearby results for hotels, vacation rentals and restaurants
  • New output field isNearbyResult
  • New input field includeNearbyResults (false by default)

2024-06-23

Fixes (tripadvisor-places)

• Fixed parsing of vacation rentals
• Removed no longer available output fields from rental results: bedroomInfo, bathroomInfo, baseDailyRate.currency

2024-06-05

Fixes (tripadvisor-places)

• Significantly improved the success rate of requests, which were being blocked by TripAdvisor's recent anti-scraping measures
• Fixed Tourism search pages crawling not working due to a change on the website

2024-05-29

Features (tripadvisor-places)

• Added extraction for TripAdvisor's "Traveler's Choice Award" badge under the new travelerChoiceAward output field

2024-05-28

Fixes (tripadvisor-places)

• Fixed location search query failing for specific less common locations

2024-05-26

Fixes (tripadvisor-places)

• Fixed restaurants price range extraction, for cases where the price range is not available and a secondary page layout
• Fixed restaurants preview photos extraction on secondary page layout
• Fixed ranking denominator data extraction on secondary restaurants page layout
• Fixed cuisines extraction for restaurants

2024-05-13

Features (tripadvisor-places)

• Added support for find restaurants URL filters, which are now used during the search
  • e.g. https://www.tripadvisor.com/FindRestaurants?geo=60763&establishmentTypes=10591&mealTypes=10598

Features (tripadvisor-reviews)

• Improved Actor to Actor integration from the full Tripadvisor Scraper to now output the scraped placeInfo data, which essentially merges the two datasets together

2024-05-04

BREAKING CHANGES (All TripAdvisor Actors)

• Added custom "error" results in cases, where the input configuration results in 0 results found. For example:
  • inputted a location that doesn't exist
  • scrape reviews for a place that has no reviews
  • scrape review in only a specific language, but the place has no reviews in that language

2024-04-28

Fixes (tripadvisor-places)

• Fixed extraction of price range and photos for restaurants
• Fixed extraction of ranking denominator data

Fixes (tripadvisor-reviews)

• Fixed review owner response extraction

2024-04-24

Hot fixes (All TripAdvisor Actors)

• Fixed crawling issues on various pages caused by a major Tripadvisor update

Hot fixes (tripadvisor-places)

• Fixed restaurant search pages crawling, also caused by a Tripadvisor update

2024-04-19

Fixes (tripadvisor-reviews)

• Fixed specific reviews which were being incorrectly translated to English

2024-03-29

Hot fixes (tripadvisor-places)

• Fixed extraction and crawling for location search pages (caused by a Tripadvisor update)
• Improved blocking rate for tourism search pages

2024-03-24

Features (tripadvisor-places)

• Added extraction of property detail preview photos under the photos field
  • added for hotels, restaurants, and attractions

Fixes (tripadvisor-places)

• Fixed handling of a very rare TripAdvisor server error, which was causing the Actor to stop crawling

2024-03-22

Fixes (tripadvisor-places)

• Fixed extraction of hotel's category review scores (categoryReviewScores) for certain hotel pages
• Fixed crawling of tourism search pages

2024-02-25

Fixes (tripadvisor-places)

• Fixed extraction of ranking data for hotels and restaurants
• Fixed extraction of restaurants price range data

2024-02-14

Hot fixes (tripadvisor-places)

• Fixed restaurant search pages failing to crawl. This was caused by Tripadvisor's updated blocking.

2024-02-09

Fixes (tripadvisor-places)

• Fixed categoryReviewScores extraction not working
• Fixed listing pages input URLs offsets being incorrectly updated if the URL parameter was already present

2024-02-08

Hot fixes (tripadvisor-places)

• Fixed hotel and vacation rentals listing page extraction issues caused by Tripadvisor's update

2024-01-30

Features (tripadvisor-places)

• Added extraction of hotel's WhatsApp redirect URL - whatsAppRedirectUrl field
• Added extraction of location's AI generated reviews summary - aiReviewsSummary field

2023-12-25

Features (tripadvisor-places)

• Added detection and handling of TripAdvisor's blocking for specific location detail pages
• Added support for a different page layout of restaurant listings

2023-12-13

Fixes (tripadvisor-places)

• Fixed locationFullName input field search not working for certain locations

2023-12-11

Fixes (tripadvisor-places)

• Fixed a few bugs related to the extraction logic of 'vacation rentals' search/listings pages.

2023-12-07

Features (tripadvisor-places)

• Implemented back an updated version of the overcome 3000 results limit feature. The current version is based on a different approach and doesn't get all the results, but still gets a way larger portion than the default search.

2023-12-05

Hot fixes (tripadvisor-places)

• Removed the overcome 3000 results limit feature, because Tripadvisor has disabled their functionality that allowed us to overcome this limit.

2023-11-30

Fixes (tripadvisor-places)

• Fixed crawling of vacation rentals search pages (caused by a Tripadvisor update)

2023-11-27

Fixes (tripadvisor-reviews)

• Fixed detection of last review pages for rare cases, where Tripadvisor was displaying incorrect number of reviews

Fixes (tripadvisor-places)

• Fixed incorrect input validation of checkInDate and checkOutDate fields

2023-11-24

Features (tripadvisor-places)

• Added support for relative dates in checkInDate and checkOutDate input fields (e.g. 3 days, 1 week or 3 months)
  • note: you need to use the JSON input editor to set relative dates

Features (tripadvisor-reviews)

• Added support for relative dates in lastReviewDate input field (e.g. 3 days, 1 week or 3 months)
  • note: you need to use the JSON input editor to set relative dates
• Added support for Actor to Actor integration from the full Tripadvisor Scraper
  • Automatically takes the scraped place URLs from the dataset of the full scraper and appends them to the startUrls input field

2023-11-20

Fixes (All TripAdvisor Actors)

• Fixed pagination offsets for certain hotel pages sometimes not working

2023-11-13

Fixes (tripadvisor-reviews)

• Fixed Actor not exiting after extracting all reviews

2023-11-11

Fixes (tripadvisor-reviews)

• Fixed the default value for lastReviewDate input field
• Fixed duplicates caused by English reviews reaching the lastReviewDate limit with 'all review languages' filter

2023-10-28

Fixes (All TripAdvisor Actors)

• Fixed extraction issues related to the Chinese language

2023-10-27

BREAKING CHANGES (tripadvisor-reviews)

• Changed/removed a few fields in the output. Most of these fields shouldn't significantly impact most users. If any of these fields are missed, please open a new issue for further consideration.

Features (tripadvisor-reviews)

• Added review rating filters (reviewRatings)
• Added review language filter for 'All languages' (reviewLanguages)
• Added functionality to select multiple review languages filters simultaneously (reviewLanguages)
• Implemented extraction of review's trip type (tripType), room tip (roomTip) and the job title of the place owner respondent (ownerResponse.connectionToSubject)
• Improved the reviews-saving process to work continuously, instead of one large batch after scraping all reviews for a specific place

Fixes (tripadvisor-reviews)

• Fixed review language filter targeting unwanted languages when a hotel lacked reviews in the desired language

2023-10-24

Features (tripadvisor-places)

• Added 'max items limit per query' input option (maxItemsPerQuery)
  • Limits the number of items that will be extracted per each start URL or search query

2023-10-13

Fixes (All TripAdvisor Actors)

• Fixed and adjusted to Tripadvisor's update that broke our scraping for certain pages

Other (All TripAdvisor Actors)

• Set Actor memory to 2 GB - this value provides optimal performance and cost

2023-10-07

Features (tripadvisor-places)

• Added 'Category review scores' extraction for hotels - categoryReviewScores field
  • e.g.: [{"categoryName": "Location", "score": 4.9}, {"categoryName": "Cleanliness", "score": 5}, ...]

2023-10-04

Fixes (tripadvisor-places)

• Fixed our duplicate results detection system getting stuck on listing pages with all search results being duplicates

2023-10-02

Fixes (tripadvisor-places)

• Fixed numberOfRooms field extraction for hotels

2023-09-22

Fixes (tripadvisor-places)

• Fixed 'thing to do' search stopping after 1000 results

2023-09-22

Fixes (tripadvisor-places)

• Fixed public trip search

2023-09-12

Fixes (tripadvisor-places)

• Skipping of the first pagination page for FindRestaurants start URLs

2023-8-29

Features (tripadvisor-reviews)

• Added place rating histogram to get the distribution of ratings for a place
• Added average place rating

2023-08-28

Fixes (All TripAdvisor Actors)

• Crashes caused by a change in scraping Tripadvisor's hotel price API
• Errors on landing pages that miss some web listing categories

Features (tripadvisor-places)

• Deep city search, use it to overcomes Tripadvisor's 3000 results limit!
  • To activate it set 'Max search results' above 3000

BREAKING CHANGES (All TripAdvisor Actors)

• Removed Proxy configuration options

2023-08-16

Features (tripadvisor-reviews)

• Added numberOfReviews to output under placeInfo field

2023-07-28

Fixes (tripadvisor-places)

• Switched to a full search instead of recommended locations when using locationFullName

2023-07-04

Fixes (tripadvisor-places)

• Localization of vacation rentals
• Localization of restaurant features

2023-05-26

Features (tripadvisor-places)

• Vacation rentals (input includeVacationRentals)
  • Separate rentals category with sublocations
  • Rental listings among hotels
• URLs with filters
  • attractions, restaurants, hotels, vacation rentals
  • tours, tours itinery, skip the line (filtered attractions)
• URLs with a single attraction and vacation rental
• Listings from public trips
  • E.g. https://www.tripadvisor.com/Trips/122547544
• Restaurant features
• Extended output

Features (tripadvisor-reviews)

• Added language selection

Fixes (tripadvisor-places)

• Less results than promoted on the website (due to using an old API)
• Inconsistent output schema (attractions were handled separately)
• Search by location using start URL
  • e.g. https://www.tripadvisor.in/Tourism-g143029-Grand_Teton_National_Park_Wyoming-Vacations.html
• Number of extracted hotels was often very different from total count displayed on the website
  • Vacation rentals are included among hotel listings even though they have their own category on the website
• Hotel prices
• Slow hotel runs