Trustpilot Company List Scraper

The Actor exports the complete list of companies on Trustpilot: 874,063 profiles, as the exact keys the other four Trustpilot Actors take as input. It is the starting point for any piece of work whose scope is "the whole site" rather than one category or one search phrase.

The boundary matters and is worth stating before anything else: this returns names and nothing else. There is no score, no review count and no location on a row. What it gives is a complete population rather than a sample, which is what a market study, a sampling frame or a bulk enrichment job needs and what a keyword search cannot produce. Feeding the domain column into Trustpilot Company Scraper then attaches the scores, the reply behaviour and the contact details to as many of them as the budget allows.

Where companies are wanted with their numbers already attached, Trustpilot Category Scraper and Trustpilot Search Scraper are the right Actors instead.

Accepted input

maxItems caps how many company keys are saved and defaults to 1000; a value of 0 lets the run continue to the end of the index. shard selects which slice of the index to start from, 0 to 17, and defaults to 0. offset selects where to start inside that slice and defaults to 0.

{
  "maxItems": 1000,
  "shard": 0,
  "offset": 0
}

shard and offset are resume points rather than filters. A run started at a given slice continues into the following slices on its own and stops at maxItems.

Response fields

{ "domain": "www.budgetair.com", "shard": 0, "position": 1 }
{ "domain": "happen.com", "shard": 0, "position": 2 }

shard and position together are a stable address within the index, and they are what a later run resumes from.

Resuming a long walk

The run log ends by naming the settings that continue the walk:

Continue this walk with Start Shard 3 and Start Offset 12000

Entering those values as shard and offset on the next run resumes exactly where the previous one stopped; nothing is re-read and nothing is skipped. A full export of 874,063 keys is a long run whatever cap is set, and this is how it is split across several.

Questions

Where does the figure 874,063 come from, and is it an estimate? It was counted. Trustpilot publishes its index in 18 slices; slices 0 to 16 hold exactly 50,000 companies each and slice 17 holds 24,063, which is why the total is 874,063 rather than 900,000. The run does not assume that division: it reads the number of slices and the length of the slice it is currently reading from each response, so the walk stays correct if Trustpilot publishes a nineteenth slice or the final slice grows. A window requested past the end of a slice returns an empty result rather than an error, and the run moves on to the next slice.

Why does a key here cost so much less than a row from the other Trustpilot Actors? Because the source is different in kind. The index is a published file that is read once and paged out of memory, whereas every row from the other Actors comes from reading a company or directory page. Pricing a name from a cached file at the same rate as a page fetch would not have been honest, so it is priced far below them.

Why do some keys contain a slash, and can they be trimmed? They cannot. Roughly one key in four hundred is a hostname followed by a path. Counted across three slices, which is 150,000 real keys, 371 carried a path: rac.co.uk/insurance, lloydsbank.com/insurance.html and mercedes-benz.co.uk/passengercars/finance.html among them. rac.co.uk/insurance and rac.co.uk are different companies with different review counts, and both return a real record, so a key truncated at the first slash does not fail; it silently returns the other company. A small number of keys are internationalised and published un-punycoded: 13 of those same 150,000, including kørkort.dk, lähitapiola.fi and colisprivé.fr. Both kinds are accepted unchanged by the other Trustpilot Actors, and neither should be trimmed or converted first.

Why do some keys start with www. and others not? Because Trustpilot files them that way, and the prefix is part of the spelling rather than decoration: trustpilot.com/review/amazon.com redirects to /review/www.amazon.com. That is much of the value of having the index at all; every key here is the canonical spelling, so nothing downstream has to guess.

What happens with a shard above 17? The input accepts 0 to 17, which is what the index currently holds, so a higher value is rejected before any request is made rather than after one.

Is a key from this list guaranteed to still have a profile? No. The index reflects what Trustpilot published when the file was generated, and a company can close or be removed afterwards. Keys that no longer resolve are reported and skipped by Trustpilot Company Scraper rather than ending its run, so a list of five hundred that contains a few dead keys still returns the rest.

Does the index cover companies outside the English-language site? Not necessarily. The index read here is the English-language one, which implies siblings for other locales and therefore more than 874,063 profiles worldwide. Those have not been measured, so no claim is made about them.

Can this be chained to the other Actors automatically? Not within one run. domain from this dataset is passed to Trustpilot Company Scraper or Trustpilot Reviews Scraper as their domain input, which is a list, so the two steps chain through the Apify API with no string manipulation in between.

Related Actors

Trustpilot Company Scraper turns keys from this list into full company records. Trustpilot Search Scraper is the better starting point where a company is known by name rather than needed in bulk.