Travel Review Aggregator — Apify Actor

Scrape hotel, restaurant, and attraction reviews, ratings, pricing, and ranking data from TripAdvisor and Booking.com.

Built for hospitality reputation management, competitive analysis, and travel market research.

---

Features

• ✅ TripAdvisor — parse Hotel_Review, Restaurant_Review, and Attraction_Review pages
• ✅ Booking.com — parse property pages (booking.com/hotel/…)
• ✅ JSON-LD structured data — primary extraction path (rich, machine-readable)
• ✅ Meta tag fallback — OG/Twitter/standard meta tags when JSON-LD is absent
• ✅ HTML review parsing — extract whatever reviews are server-side rendered
• ✅ User-agent rotation — 10 modern desktop + mobile agents
• ✅ Polite crawling — 2–4s random delays between requests
• ✅ Retry with exponential backoff — max 3 retries on 403/429/network errors
• ✅ Graceful abort — clean shutdown on Apify platform abort signal
• ✅ Concurrency control — semaphore-limited (max 3 concurrent requests)
• ✅ Apify dataset output — results pushed to the default dataset

---

Input

Property	Type	Default	Description
tripadvisorUrls	string[]	[]	List of TripAdvisor listing page URLs
bookingUrls	string[]	[]	List of Booking.com property page URLs
maxReviews	integer	50	Max reviews to extract per listing (server-side only)
includeReviewerInfo	boolean	true	Whether to include reviewer names in output

Example Input (JSON)

{
  "tripadvisorUrls": [
    "https://www.tripadvisor.com/Hotel_Review-g60763-d122439-Reviews-New_York_Marriott_Marquis-New_York_City_New_York.html",
    "https://www.tripadvisor.com/Restaurant_Review-g60763-d4256615-Reviews-ABC_Kitchen-New_York_City_New_York.html"
  ],
  "bookingUrls": [
    "https://www.booking.com/hotel/us/new-york-marriott-marquis.html"
  ],
  "maxReviews": 50,
  "includeReviewerInfo": true
}

---

Output

Each listing produces a dataset item with the following fields:

Field	Type	Description
source	string	"tripadvisor" or "booking"
name	string	Listing name
url	string	Listing URL
pageType	string	"hotel", "restaurant", "attraction", or "unknown"
address	string	Full address
latitude	number	Geographic latitude
longitude	number	Geographic longitude
rating	number	Overall aggregate rating
totalReviews	number	Total review count
ranking	string	TripAdvisor ranking text (e.g. "#3 of 1,200 things to do in Paris")
priceRange	string	Price level or range
starRating	number	Hotel star rating (Booking.com)
telephone	string	Phone number
description	string	Listing description
servesCuisine	string[]	Cuisine types (restaurants)
amenities	string[]	Amenities / features
reviewScore	object	Booking.com score breakdown
reviews	array	Array of review objects
breadcrumbs	string[]	Breadcrumb navigation
scrapedAt	string	ISO-8601 scrape timestamp

Review Object

Field	Type	Description
author	string	Reviewer name (null when includeReviewerInfo is false)
rating	number	Individual review rating
date	string	Review date
title	string	Review title
body	string	Review body text
language	string	Review language code
stayDate	string	Stay date (Booking.com only)
roomType	string	Room type (Booking.com only)
travelerType	string	Traveler type (Booking.com only)

---

How It Works

Scraping Strategy

1. Primary — JSON-LD: Parse <script type="application/ld+json"> blocks for structured data (business info, aggregate ratings, individual reviews).
2. Secondary — Meta tags: Extract OG (og:title, og:description), Twitter, and standard meta tags as fallback.
3. Tertiary — HTML parsing: Parse visible HTML for review cards, amenity tags, pricing, and ranking information not available in structured data.

Anti-Scraping Measures

Both TripAdvisor and Booking.com actively protect their content. This actor implements:

• User-agent rotation — 10 modern browser user agents, randomly selected per request
• Realistic delays — 2–4 seconds between requests (configurable via MIN_DELAY / MAX_DELAY)
• Retry with backoff — up to 3 retries with exponential backoff on 403 (Forbidden), 429 (Too Many Requests), and network errors
• Polite concurrency — maximum 3 concurrent requests via asyncio.Semaphore
• Realistic headers — Accept-Language, Sec-Fetch-*, and other browser-like headers

---

Limitations

Server-Side Rendering

Both TripAdvisor and Booking.com load the majority of reviews dynamically via JavaScript. This actor uses plain HTTP requests (not a headless browser), so only reviews present in the initial server-side HTML or JSON-LD blocks will be captured — typically 3–10 reviews per page, not the full list.

If you need all reviews, consider:

• Using Apify's PuppeteerCrawler or PlaywrightCrawler with this code as a base
• Paginating through review pages (TripAdvisor offers -oa30- offset URLs, Booking.com uses API calls)

Anti-Scraping Blocks

• TripAdvisor may return 403 or a CAPTCHA page under aggressive crawling
• Booking.com may return 403 or serve a cookie-consent wall
• The built-in delays and retries mitigate this but cannot guarantee access

Data Freshness

Ratings, prices, and review counts are as-of-the-request snapshots and may change.

---

Local Development

# Install dependencies
pip install -r requirements.txt

# Run with Apify CLI
apify run --purge

Requirements

• Python 3.12+
• apify ≥ 3.0
• beautifulsoup4 with lxml
• httpx ≥ 0.28

---

Docker

The included Dockerfile builds a production image using apify/actor-python:3.14.

docker build -t travel-review-aggregator .
docker run travel-review-aggregator

---

License

MIT