Tripadvisor Scraper

This changelog summarizes all changes of the TripAdvisor Actors provided by Maximillian Copelli. The specific Actors that are affected are listed for each change.

2024-04-24

Hot fixes (All TripAdvisor Actors)

• Fixed crawling issues on various pages caused by a major Tripadvisor update

Hot fixes (tripadvisor-places)

• Fixed restaurant search pages crawling, also caused by a Tripadvisor update

2024-04-19

Fixes (tripadvisor-reviews)

• Fixed specific reviews which were being incorrectly translated to English

2024-03-29

Hot fixes (tripadvisor-places)

• Fixed extraction and crawling for location search pages (caused by a Tripadvisor update)
• Improved blocking rate for tourism search pages

2024-03-24

Features (tripadvisor-places)

• Added extraction of property detail preview photos under the photos field
  • added for hotels, restaurants, and attractions

Fixes (tripadvisor-places)

• Fixed handling of a very rare TripAdvisor server error, which was causing the Actor to stop crawling

2024-03-22

Fixes (tripadvisor-places)

• Fixed extraction of hotel's category review scores (categoryReviewScores) for certain hotel pages
• Fixed crawling of tourism search pages

2024-02-25

Fixes (tripadvisor-places)

• Fixed extraction of ranking data for hotels and restaurants
• Fixed extraction of restaurants price range data

2024-02-14

Hot fixes (tripadvisor-places)

• Fixed restaurant search pages failing to crawl. This was caused by Tripadvisor's updated blocking.

2024-02-09

Fixes (tripadvisor-places)

• Fixed categoryReviewScores extraction not working
• Fixed listing pages input URLs offsets being incorrectly updated if the URL parameter was already present

2024-02-08

Hot fixes (tripadvisor-places)

• Fixed hotel and vacation rentals listing page extraction issues caused by Tripadvisor's update

2024-01-30

Features (tripadvisor-places)

• Added extraction of hotel's WhatsApp redirect URL - whatsAppRedirectUrl field
• Added extraction of location's AI generated reviews summary - aiReviewsSummary field

2023-12-25

Features (tripadvisor-places)

• Added detection and handling of TripAdvisor's blocking for specific location detail pages
• Added support for a different page layout of restaurant listings

2023-12-13

Fixes (tripadvisor-places)

• Fixed locationFullName input field search not working for certain locations

2023-12-11

Fixes (tripadvisor-places)

• Fixed a few bugs related to the extraction logic of 'vacation rentals' search/listings pages.

2023-12-07

Features (tripadvisor-places)

• Implemented back an updated version of the overcome 3000 results limit feature. The current version is based on a different approach and doesn't get all the results, but still gets a way larger portion than the default search.

2023-12-05

Hot fixes (tripadvisor-places)

• Removed the overcome 3000 results limit feature, because Tripadvisor has disabled their functionality that allowed us to overcome this limit.

2023-11-30

Fixes (tripadvisor-places)

• Fixed crawling of vacation rentals search pages (caused by a Tripadvisor update)

2023-11-27

Fixes (tripadvisor-reviews)

• Fixed detection of last review pages for rare cases, where Tripadvisor was displaying incorrect number of reviews

Fixes (tripadvisor-places)

• Fixed incorrect input validation of checkInDate and checkOutDate fields

2023-11-24

Features (tripadvisor-places)

• Added support for relative dates in checkInDate and checkOutDate input fields (e.g. 3 days, 1 week or 3 months)
  • note: you need to use the JSON input editor to set relative dates

Features (tripadvisor-reviews)

• Added support for relative dates in lastReviewDate input field (e.g. 3 days, 1 week or 3 months)
  • note: you need to use the JSON input editor to set relative dates
• Added support for Actor to Actor integration from the full Tripadvisor Scraper
  • Automatically takes the scraped place URLs from the dataset of the full scraper and appends them to the startUrls input field

2023-11-20

Fixes (All TripAdvisor Actors)

• Fixed pagination offsets for certain hotel pages sometimes not working

2023-11-13

Fixes (tripadvisor-reviews)

• Fixed Actor not exiting after extracting all reviews

2023-11-11

Fixes (tripadvisor-reviews)

• Fixed the default value for lastReviewDate input field
• Fixed duplicates caused by English reviews reaching the lastReviewDate limit with 'all review languages' filter

2023-10-28

Fixes (All TripAdvisor Actors)

• Fixed extraction issues related to the Chinese language

2023-10-27

BREAKING CHANGES (tripadvisor-reviews)

• Changed/removed a few fields in the output. Most of these fields shouldn't significantly impact most users. If any of these fields are missed, please open a new issue for further consideration.

Features (tripadvisor-reviews)

• Added review rating filters (reviewRatings)
• Added review language filter for 'All languages' (reviewLanguages)
• Added functionality to select multiple review languages filters simultaneously (reviewLanguages)
• Implemented extraction of review's trip type (tripType), room tip (roomTip) and the job title of the place owner respondent (ownerResponse.connectionToSubject)
• Improved the reviews-saving process to work continuously, instead of one large batch after scraping all reviews for a specific place

Fixes (tripadvisor-reviews)

• Fixed review language filter targeting unwanted languages when a hotel lacked reviews in the desired language

2023-10-24

Features (tripadvisor-places)

• Added 'max items limit per query' input option (maxItemsPerQuery)
  • Limits the number of items that will be extracted per each start URL or search query

2023-10-13

Fixes (All TripAdvisor Actors)

• Fixed and adjusted to Tripadvisor's update that broke our scraping for certain pages

Other (All TripAdvisor Actors)

• Set Actor memory to 2 GB - this value provides optimal performance and cost

2023-10-07

Features (tripadvisor-places)

• Added 'Category review scores' extraction for hotels - categoryReviewScores field
  • e.g.: [{"categoryName": "Location", "score": 4.9}, {"categoryName": "Cleanliness", "score": 5}, ...]

2023-10-04

Fixes (tripadvisor-places)

• Fixed our duplicate results detection system getting stuck on listing pages with all search results being duplicates

2023-10-02

Fixes (tripadvisor-places)

• Fixed numberOfRooms field extraction for hotels

2023-09-22

Fixes (tripadvisor-places)

• Fixed 'thing to do' search stopping after 1000 results

2023-09-22

Fixes (tripadvisor-places)

• Fixed public trip search

2023-09-12

Fixes (tripadvisor-places)

• Skipping of the first pagination page for FindRestaurants start URLs

2023-8-29

Features (tripadvisor-reviews)

• Added place rating histogram to get the distribution of ratings for a place
• Added average place rating

2023-08-28

Fixes (All TripAdvisor Actors)

• Crashes caused by a change in scraping Tripadvisor's hotel price API
• Errors on landing pages that miss some web listing categories

Features (tripadvisor-places)

• Deep city search, use it to overcomes Tripadvisor's 3000 results limit!
  • To activate it set 'Max search results' above 3000

BREAKING CHANGES (All TripAdvisor Actors)

• Removed Proxy configuration options

2023-08-16

Features (tripadvisor-reviews)

• Added numberOfReviews to output under placeInfo field

2023-07-28

Fixes (tripadvisor-places)

• Switched to a full search instead of recommended locations when using locationFullName

2023-07-04

Fixes (tripadvisor-places)

• Localization of vacation rentals
• Localization of restaurant features

2023-05-26

Features (tripadvisor-places)

• Vacation rentals (input includeVacationRentals)
  • Separate rentals category with sublocations
  • Rental listings among hotels
• URLs with filters
  • attractions, restaurants, hotels, vacation rentals
  • tours, tours itinery, skip the line (filtered attractions)
• URLs with a single attraction and vacation rental
• Listings from public trips
  • E.g. https://www.tripadvisor.com/Trips/122547544
• Restaurant features
• Extended output

Features (tripadvisor-reviews)

• Added language selection

Fixes (tripadvisor-places)

• Less results than promoted on the website (due to using an old API)
• Inconsistent output schema (attractions were handled separately)
• Search by location using start URL
  • e.g. https://www.tripadvisor.in/Tourism-g143029-Grand_Teton_National_Park_Wyoming-Vacations.html
• Number of extracted hotels was often very different from total count displayed on the website
  • Vacation rentals are included among hotel listings even though they have their own category on the website
• Hotel prices
• Slow hotel runs