Trustpilot reviews scraper

Introduction

Collecting reviews from companies on trustpilot.com becomes a breeze using this product. You can easily filter for the specific reviews you need and save them to Excel, CSV, XML, JSON and other formats. You can also manage the collection of reviews using an API if you are tech savvy.

Certain graphics has been blurred for copyright reasons

Use cases

Common use cases for collecting reviews on Trustpilot are to:

• Display the reviews on your own site
• Analyze your competitor's reviews to determine why customers are happy or unhappy with the company
• Analyze how your competitor responds to reviews
• Determine you target audience and analyze your customers profile
• Monitor mentions of your brand
• Find new leads
• Do market research
• Do research on behavior of reviewers or amount of fake reviews for example by analysis with machine learning

How to use

Here is an example of the inputs:

Fill out the following inputs when running the scraper:

[companyWebsite]: Enter the website for the company you want reviews for, such as shopwagandtail.com. To collect reviews for multiple companies you can setup an Apify task. Let me know if you need help with this.

[onlyExtractCompanyInformation]: If "true" is set in the text, the scraper will only collect information about the company on the review page and not collect any reviews. If it is left as "false" it will collect both company information and reviews

[sortBy]: Optional sorting by recent reviews. If you don't specify this, sorting will be done by relevancy.

[filterByStarRating]: Optional filtering by amount of stars for a review. You can specify either 1,2,3,4 or 5. However you can not filter by multiple stars.

[filterByLanguage]: Optional filtering by languauge of reviews. You can only specify only language, such as es, da and so on. By default all languages are selected.

[filterByCountry]: Optional filtering by country of the reviewer. You can only specify one country, such as "United States" which would be the ISO country code "US". Casing must match. By default all countries are selected. A list of country codes can be found here: https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2",

[filterByVerified]: Optional filtering by if reviews have been verified, such as when the company has asked the customer to do a review. Type true to enable this filter.

It is recommended to use proxies for scraping many reviews at a time.

Change timeout setting to facilitate scraping the amount of reviews you need. A default timeout is set for 3600 seconds (1 hour), which is sufficient for scraping all reviews from a small company such as shopwagandtail.com and bigger companies as well. The required timeout depends entirely on your use case and the amount of reviews you will collect. Let me know if you need help estimating this by opening an issue.

Optionally set a different memory amount. By default the scraper runs optimal with a minimimum of 4GB of memory. The performance of the scraper when running with recommended configuration with proxies and 4GB of memory is the ability to scrape approximately on average 3 reviews/second and 5 seconds to process each page. However these are estimates as trustpilot.com can throttle how many pages can be processed at a given speed. Use proxies to circumvent this.

Extracted data (datasets)

These are the datasets that contain results from running the scraper (see explanation in the below section for each dataset):

• CompanyInformation dataset
• Reviews dataset

Note that the datasets (company information and company reviews) can be found in your apify.com control panel => Storage => Datasets tab. These datasets are named and will not expire.

The extracted data will be for the CompanyInformation dataset:

• Official count of reviews on Trustpilot (this number is not accurate as reviews might await processing, be hidden or deleted as per Trustpilot algorithms)
• Category of the company
• Company name
• Page URL of the company on Trustpilot
• Whether or not the company is verified by Trustpilot
• Average rating score for all reviews for the company
• The Business Unit ID that identifies the company the reviews was scraped from
• When the data about the company was scraped from trustpilot.com

The extracted data will for the reviews dataset be:

• The title of the review
• The description/content of the review
• ID of the review
• The rating score in stars that the reviewer has given the review
• The reviewers country
• The response from the company, if they have written any
• The name of the reviewer
• If the review is verified
• The language of the review, if it is found, otherwise the value will be "unknown language"
• The date and time of the review
• URL that points to the page to find the review
• The page number that a review was scraped from on trustpilot.com
• When the review was scraped
• The Business Unit ID that identifies the company the reviews was scraped from

Here is a sample of company information for a company on Trustpilot:

1[
2  {
3    "company": "Wag + Tail ",
4    "rating": "4.8",
5    "OfficialTotalReviewCount": 194,
6    "isCompanyVerified": "yes",
7    "category": "Pet Store"
8  }
9]

Here is a sample Trustpilot review from a sample company on Trustpilot:

1[
2  {
3    "companyPageUrl": "https://www.trustpilot.com/review/shopwagandtail.com",
4    "reviewId": "62d02e3a8000af4a88401350",
5    "companyName": "Wag + Tail",
6    "businessUnitId": "624c24851220e2743a4d7916",
7    "reviewUrl": "https://trustpilot.com/reviews/62d02e3a8000af4a88401350",
8    "reviewDate": "2022-07-14T18:54:50.000Z",
9    "isReviewVerified": false,
10    "reviewer": "Maryanne Ramirez",
11    "reviewTitle": "Cutest Bandanas Ever!",
12    "reviewDescription": "I absolutely love this brand! Their bandanas are beautifully designed and are of great quality! Super easy to tie on my puppies and they look amazing on them. 10/10 I would definitely recommend these products!",
13    "reviewRatingScore": 5,
14    "reviewersCountry": "Unknown",
15    "reviewLanguage": "uk",
16    "reviewCompanyResponse": "Hi Maryanne,\n\nWe are thrilled to have you as part of the Wag and Tail family! Thank you for sharing your kind words with us. It means the world to us and to our entire team! We look forward to serving you again soon! 🧡\n\nBest Regards,\nTeam Wag and Tail",
17    "scrapedDateTime": "2024-02-26T11:43:10.545Z",
18    "scrapedAtReviewPageNumber": 2
19  }
20]

Some graphics has been blurred for copyright reasons

Known limitations

• Reviews saved to CSV can in some cases look a bit messy. Let me know if you experience that.

• The Apify API has a maximum timeout of 300 seconds, thus it is advised to create an async API request to initiate the scraping run and then setup a webhook in Apify that will send an API request to your system when the processing is done. Let me know if you need help setting this up.

• This actor is not intended for extracting data that will be displayed realtime on the frontend to a client because it may take several seconds (minimum of 15 seconds) to minutes to retrieve the data - Extract the data ahead of time and store in your own database to easily present in your frontend. This can be done through the Apify API.

Breaking changes

Prevent being impacted by breaking changes by selecting a specific version and build to use rather than the latest build

Help, feedback and bugs

Let me know if you need some other data and I will do my best to fullfill your request.

If you need any help, post processing of the data or encounter any issues or bugs, please open an issue on apify.com at https://console.apify.com/actors/l3wcDhSSC96LBRUpc/issues and I will get back to you as soon as possible. You can also contact me on my socials: Linkedin: https://www.linkedin.com/in/casper-rubaek/ and discord: casper#1906 or email: casper@fambm.dk. Do also feel free to join my Discord community CaspersApifyCommunity to engage with other customers and me to enhance your experience. You can join via this link: https://discord.gg/hD5ZMncS5U

Legal disclaimer

Please note that this product is not a product of Trustpilot, anyhow affiliated with or endorsed by Trustpilot.

As a user of this webscraping solution, you are solely responsible for complying with applicable laws and respecting Trustpilot' terms of use, such as copyright etc. when using the extracted review data.