# TripAdvisor & Booking.com Review Aggregator (`second_coming/travel-review-aggregator`) Actor

Scrape hotel, restaurant, and attraction reviews, ratings, pricing, and ranking data from TripAdvisor and Booking.com. Hospitality reputation management tool.

- **URL**: https://apify.com/second\_coming/travel-review-aggregator.md
- **Developed by:** [Richard P](https://apify.com/second_coming) (community)
- **Categories:** Travel, Business
- **Stats:** 1 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Travel Review Aggregator — Apify Actor

Scrape hotel, restaurant, and attraction **reviews**, **ratings**, **pricing**, and **ranking** data from **TripAdvisor** and **Booking.com**.

Built for hospitality reputation management, competitive analysis, and travel market research.

---

### Features

- ✅ **TripAdvisor** — parse `Hotel_Review`, `Restaurant_Review`, and `Attraction_Review` pages
- ✅ **Booking.com** — parse property pages (`booking.com/hotel/…`)
- ✅ **JSON-LD structured data** — primary extraction path (rich, machine-readable)
- ✅ **Meta tag fallback** — OG/Twitter/standard meta tags when JSON-LD is absent
- ✅ **HTML review parsing** — extract whatever reviews are server-side rendered
- ✅ **User-agent rotation** — 10 modern desktop + mobile agents
- ✅ **Polite crawling** — 2–4s random delays between requests
- ✅ **Retry with exponential backoff** — max 3 retries on 403/429/network errors
- ✅ **Graceful abort** — clean shutdown on Apify platform abort signal
- ✅ **Concurrency control** — semaphore-limited (max 3 concurrent requests)
- ✅ **Apify dataset output** — results pushed to the default dataset

---

### Input

| Property | Type | Default | Description |
|---|---|---|---|
| `tripadvisorUrls` | `string[]` | `[]` | List of TripAdvisor listing page URLs |
| `bookingUrls` | `string[]` | `[]` | List of Booking.com property page URLs |
| `maxReviews` | `integer` | `50` | Max reviews to extract per listing (server-side only) |
| `includeReviewerInfo` | `boolean` | `true` | Whether to include reviewer names in output |

#### Example Input (JSON)

```json
{
  "tripadvisorUrls": [
    "https://www.tripadvisor.com/Hotel_Review-g60763-d122439-Reviews-New_York_Marriott_Marquis-New_York_City_New_York.html",
    "https://www.tripadvisor.com/Restaurant_Review-g60763-d4256615-Reviews-ABC_Kitchen-New_York_City_New_York.html"
  ],
  "bookingUrls": [
    "https://www.booking.com/hotel/us/new-york-marriott-marquis.html"
  ],
  "maxReviews": 50,
  "includeReviewerInfo": true
}
````

***

### Output

Each listing produces a dataset item with the following fields:

| Field | Type | Description |
|---|---|---|
| `source` | `string` | `"tripadvisor"` or `"booking"` |
| `name` | `string` | Listing name |
| `url` | `string` | Listing URL |
| `pageType` | `string` | `"hotel"`, `"restaurant"`, `"attraction"`, or `"unknown"` |
| `address` | `string` | Full address |
| `latitude` | `number` | Geographic latitude |
| `longitude` | `number` | Geographic longitude |
| `rating` | `number` | Overall aggregate rating |
| `totalReviews` | `number` | Total review count |
| `ranking` | `string` | TripAdvisor ranking text (e.g. `"#3 of 1,200 things to do in Paris"`) |
| `priceRange` | `string` | Price level or range |
| `starRating` | `number` | Hotel star rating (Booking.com) |
| `telephone` | `string` | Phone number |
| `description` | `string` | Listing description |
| `servesCuisine` | `string[]` | Cuisine types (restaurants) |
| `amenities` | `string[]` | Amenities / features |
| `reviewScore` | `object` | Booking.com score breakdown |
| `reviews` | `array` | Array of review objects |
| `breadcrumbs` | `string[]` | Breadcrumb navigation |
| `scrapedAt` | `string` | ISO-8601 scrape timestamp |

#### Review Object

| Field | Type | Description |
|---|---|---|
| `author` | `string` | Reviewer name (null when `includeReviewerInfo` is false) |
| `rating` | `number` | Individual review rating |
| `date` | `string` | Review date |
| `title` | `string` | Review title |
| `body` | `string` | Review body text |
| `language` | `string` | Review language code |
| `stayDate` | `string` | Stay date (Booking.com only) |
| `roomType` | `string` | Room type (Booking.com only) |
| `travelerType` | `string` | Traveler type (Booking.com only) |

***

### How It Works

#### Scraping Strategy

1. **Primary — JSON-LD**: Parse `<script type="application/ld+json">` blocks for structured data (business info, aggregate ratings, individual reviews).
2. **Secondary — Meta tags**: Extract OG (`og:title`, `og:description`), Twitter, and standard meta tags as fallback.
3. **Tertiary — HTML parsing**: Parse visible HTML for review cards, amenity tags, pricing, and ranking information not available in structured data.

#### Anti-Scraping Measures

Both TripAdvisor and Booking.com actively protect their content. This actor implements:

- **User-agent rotation** — 10 modern browser user agents, randomly selected per request
- **Realistic delays** — 2–4 seconds between requests (configurable via `MIN_DELAY` / `MAX_DELAY`)
- **Retry with backoff** — up to 3 retries with exponential backoff on 403 (Forbidden), 429 (Too Many Requests), and network errors
- **Polite concurrency** — maximum 3 concurrent requests via `asyncio.Semaphore`
- **Realistic headers** — `Accept-Language`, `Sec-Fetch-*`, and other browser-like headers

***

### Limitations

#### Server-Side Rendering

Both TripAdvisor and Booking.com load the **majority of reviews dynamically via JavaScript**. This actor uses plain HTTP requests (not a headless browser), so only reviews present in the initial server-side HTML or JSON-LD blocks will be captured — typically **3–10 reviews** per page, not the full list.

**If you need all reviews**, consider:

- Using Apify's **PuppeteerCrawler** or **PlaywrightCrawler** with this code as a base
- Paginating through review pages (TripAdvisor offers `-oa30-` offset URLs, Booking.com uses API calls)

#### Anti-Scraping Blocks

- TripAdvisor may return 403 or a CAPTCHA page under aggressive crawling
- Booking.com may return 403 or serve a cookie-consent wall
- The built-in delays and retries mitigate this but cannot guarantee access

#### Data Freshness

Ratings, prices, and review counts are **as-of-the-request** snapshots and may change.

***

### Local Development

```bash
## Install dependencies
pip install -r requirements.txt

## Run with Apify CLI
apify run --purge
```

#### Requirements

- Python 3.12+
- `apify` ≥ 3.0
- `beautifulsoup4` with `lxml`
- `httpx` ≥ 0.28

***

### Docker

The included `Dockerfile` builds a production image using `apify/actor-python:3.14`.

```bash
docker build -t travel-review-aggregator .
docker run travel-review-aggregator
```

***

### License

MIT

# Actor input Schema

## `tripadvisorUrls` (type: `array`):

List of TripAdvisor listing page URLs (Hotel\_Review, Restaurant\_Review, Attraction\_Review).

## `bookingUrls` (type: `array`):

List of Booking.com property page URLs.

## `maxReviews` (type: `integer`):

Maximum number of reviews to extract per listing page. Note that server-side rendered reviews are limited — many reviews are loaded dynamically via JavaScript and may not be available via HTTP requests alone.

## `includeReviewerInfo` (type: `boolean`):

If enabled, reviewer names and details are included in the output. Disable for privacy-focused scraping.

## Actor input object example

```json
{
  "tripadvisorUrls": [
    "https://www.tripadvisor.com/Hotel_Review-g60763-d122439-Reviews-New_York_Marriott_Marquis-New_York_City_New_York.html"
  ],
  "bookingUrls": [
    "https://www.booking.com/hotel/us/new-york-marriott-marquis.html"
  ],
  "maxReviews": 50,
  "includeReviewerInfo": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("second_coming/travel-review-aggregator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("second_coming/travel-review-aggregator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call second_coming/travel-review-aggregator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=second_coming/travel-review-aggregator",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "TripAdvisor & Booking.com Review Aggregator",
        "description": "Scrape hotel, restaurant, and attraction reviews, ratings, pricing, and ranking data from TripAdvisor and Booking.com. Hospitality reputation management tool.",
        "version": "0.0",
        "x-build-id": "jBOTaFYUynyziw5u8"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/second_coming~travel-review-aggregator/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-second_coming-travel-review-aggregator",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/second_coming~travel-review-aggregator/runs": {
            "post": {
                "operationId": "runs-sync-second_coming-travel-review-aggregator",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/second_coming~travel-review-aggregator/run-sync": {
            "post": {
                "operationId": "run-sync-second_coming-travel-review-aggregator",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "properties": {
                    "tripadvisorUrls": {
                        "title": "TripAdvisor URLs",
                        "type": "array",
                        "description": "List of TripAdvisor listing page URLs (Hotel_Review, Restaurant_Review, Attraction_Review).",
                        "items": {
                            "type": "string"
                        }
                    },
                    "bookingUrls": {
                        "title": "Booking.com URLs",
                        "type": "array",
                        "description": "List of Booking.com property page URLs.",
                        "items": {
                            "type": "string"
                        }
                    },
                    "maxReviews": {
                        "title": "Maximum Reviews per Listing",
                        "minimum": 1,
                        "maximum": 500,
                        "type": "integer",
                        "description": "Maximum number of reviews to extract per listing page. Note that server-side rendered reviews are limited — many reviews are loaded dynamically via JavaScript and may not be available via HTTP requests alone.",
                        "default": 50
                    },
                    "includeReviewerInfo": {
                        "title": "Include Reviewer Information",
                        "type": "boolean",
                        "description": "If enabled, reviewer names and details are included in the output. Disable for privacy-focused scraping.",
                        "default": true
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```