# Similarweb Top Websites Scraper — Category Rankings (`scrapersdelight/similarweb-traffic-scraper`) Actor

Similarweb's public Top Websites tables as data. One row per ranked site: rank, domain, category, rank change month over month, bounce rate, pages per visit, visit duration. 209 categories and 67 countries from Similarweb's own taxonomy. 50 rows per worldwide category, 5 per country+category.

- **URL**: https://apify.com/scrapersdelight/similarweb-traffic-scraper.md
- **Developed by:** [Scrapers Delight](https://apify.com/scrapersdelight) (community)
- **Categories:** Marketing, Business, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 per ranked websites

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Similarweb Top Websites Scraper — Category Rankings

Turn [Similarweb's public Top Websites tables](https://www.similarweb.com/top-websites/) into data.
One row per ranked website: **rank, domain, the site's own category, month-over-month rank change,
isNewToRanking, avgVisitDuration, pagesPerVisit** and **bounceRatePercent**, across **209 categories**
and **67 countries** — every slug read live off Similarweb's own taxonomy, so nothing here can be
mistyped. No login, no API key, no CAPTCHA solving.

### Read the cap before you plan a run

Similarweb does not publish the same number of rows for every URL shape. All eight shapes were
measured on **2026-08-15**, through Apify RESIDENTIAL:

| What you ask for | Rows Similarweb publishes |
| --- | --- |
| Worldwide, no category (`/top-websites/`) | **50** |
| Worldwide + category (`/top-websites/finance/`) | **50** |
| Worldwide + subcategory (`/top-websites/finance/investing/`) | **50** |
| One country, no category (`/top-websites/united-states/`) | **50** |
| One country **+** category (`/top-websites/united-states/finance/`) | **5** |
| One country **+** subcategory | **5** |

**Combining a country with a category gives you 5 rows, not 50.** That is a product decision on
Similarweb's side, not a paging limit — nothing in the URL, the query string or the page's embedded
state pages past it. This Actor names every affected pair in the run log before it fetches anything,
and you are billed per row delivered, so a 5-row slice costs you a tenth of a 50-row one. If you
want 50 rows, use **Worldwide + a category**, or **a country with no category**.

Two more limits, stated up front rather than discovered in the CSV:

- **There is no visit count on this table.** Similarweb publishes rank and the three engagement
  metrics on the ranking page and nothing else for logged-out visitors. This Actor does not invent
  one. Each row carries `websiteAnalysisUrl` — the site's own Similarweb page — if you need to go
  deeper by hand.
- **These are Similarweb's modelled estimates**, from its panel and clickstream data. They are not
  any site's own analytics, and Similarweb itself labels them "estimated data" everywhere on the
  page. Treat rank and bounce rate as directional market signal, not as measured truth.

***

### The shipped defaults

```json
{
  "categories": [
    "finance",
    "computers-electronics-and-technology/programming-and-developer-software",
    "e-commerce-and-shopping/marketplace"
  ],
  "countries": ["worldwide"],
  "includeAllCategories": false,
  "maxItems": 500
}
```

Click **Try for free** and hit **Start** — that block is literally the input the Actor ships with.
A run on those defaults returned **150 rows in 11 seconds, from 3 slices, on 3 HTTP calls**, and
costs **$0.15**.

A larger validation run on **2026-08-15** — 22 categories × 2 countries (worldwide + United States),
including both overall rankings — returned **1,255 rows from 44 of 44 slices in 3 minutes 3 seconds
on 45 HTTP calls** (one transient proxy retry), with **zero duplicates**.

***

### What a row looks like

```json
{
  "rankingScopeCountry": "Worldwide",
  "rankingScopeCountryCode": null,
  "rankingScopeCategory": "Finance",
  "rankingScopeCategorySlug": "finance",
  "dataMonth": "July 2026",
  "snapshotDate": "2026-07-01",
  "rank": 1,
  "domain": "paypal.com",
  "siteCategory": "Finance > Banking Credit and Lending",
  "siteCategoryId": "finance/banking_credit_and_lending",
  "rankChange": 0,
  "isNewToRanking": false,
  "avgVisitDuration": "00:02:23",
  "avgVisitDurationSeconds": 143,
  "pagesPerVisit": 4.96,
  "bounceRatePercent": 21.2,
  "websiteAnalysisUrl": "https://www.similarweb.com/website/paypal.com/",
  "rankingUrl": "https://www.similarweb.com/top-websites/finance/",
  "faviconUrl": "https://site-images.similarcdn.com/image?url=paypal.com&t=2&s=1&h=…",
  "dataSource": "Similarweb public estimates",
  "scrapedAt": "2026-08-15T16:37:20.191Z"
}
```

#### Field fill — measured, not estimated

Measured on the full **1,255-row** validation run above (22 categories × 2 countries), sorted
descending. Sparse fields are **bold**.

| Field | Fill | Note |
| --- | --- | --- |
| `rank` | 100% | position inside the slice you asked for |
| `domain` | 100% | |
| `siteCategory` / `siteCategoryId` | 100% | the site's OWN main category, which is often a different leaf from the slice it was listed under |
| `rankChange` | 100% | positions gained vs last month; `0` on 29.7% of rows, non-zero on 70.3% |
| `avgVisitDuration` / `avgVisitDurationSeconds` | 100% | |
| `pagesPerVisit` | 100% | |
| `bounceRatePercent` | 100% | |
| `dataMonth` / `snapshotDate` | 100% | Similarweb's monthly snapshot, not the scrape date |
| `websiteAnalysisUrl` / `rankingUrl` / `faviconUrl` | 100% | |
| `rankingScopeCountry` / `rankingScopeCategory` | 100% | echoes back what you asked for, so a merged CSV stays sortable |
| **`rankingScopeCategorySlug`** | **92%** | null on the "overall ranking" slices, which have no category |
| **`rankingScopeCountryCode`** | **12.4%** | null for the Worldwide scope, which has no country |
| **`isNewToRanking`** | **1.0% true** | Similarweb's own "new entry" flag — 12 of 1,255 rows. A genuinely rare signal, and the most interesting column in the file if you are hunting risers |
| **monthly visits** | **absent** | not published on this table at all — see the cap section above |

***

### Who buys this

- **Competitive intel and agency pitch decks** — "here are the top 50 sites in your category, with
  engagement benchmarks, and here is where you sit." `bounceRatePercent`, `pagesPerVisit` and
  `avgVisitDurationSeconds` across a whole category give you the benchmark line that makes a
  category chart mean something.
- **M\&A and investor screening** — 209 categories × 50 rows is a screenable universe of category
  leaders with a month-over-month rank delta on every one.
- **Ad-tech and B2B prospecting** — a ranked, category-tagged list of the biggest sites in a vertical
  is a target list; `websiteAnalysisUrl` is the drill-down.
- **Rank tracking over time** — run it monthly on a schedule and diff `rank` and `rankChange`. The
  `isNewToRanking` flag surfaces entrants without you having to diff anything.

`rankChange` and `isNewToRanking` are the columns you cannot rebuild yourself from a single pull:
they are Similarweb's own month-over-month comparison, already computed.

***

### How it gets the data

The ranking pages server-render their whole table into `window.__APP_DATA__` before React runs, so
this Actor reads the app's own state object rather than scraping styled DOM nodes. That is both more
precise — the rendered table prints `23.85%` and `8.43`, the state carries `0.23845442795516247` and
`8.429506649675165` — and more stable, because the table's class names are build artefacts of a
versioned bundle while the state keys are the app's contract.

**Transport, measured 2026-08-15 through Apify:**

- This route carries **no bot wall**: plain HTTP, no `x-amzn-waf-action` header, no token, no
  browser. **44 of 44 slices succeeded** on the validation run.
- **Datacenter proxies are refused.** 13 of 30 datacenter calls came back a bare HTTP 403 from
  CloudFront (56.7% success), so **Apify RESIDENTIAL is the default** and you should leave it alone.
- The Actor retries a slice up to 3 times with a fresh residential session before giving up, and a
  slice it could not read is **never billed**.
- It runs in **512 MB** and needs no Chromium.

*(For contrast, and so you are not surprised: Similarweb's per-domain pages under `/website/` are a
different story — those sit behind an AWS WAF that escalates to a visual CAPTCHA. This Actor does not
touch them. It only reads the ranking tables, which are open.)*

#### robots.txt

`https://www.similarweb.com/robots.txt`, fetched 2026-08-15, contains these `Disallow` lines for
`User-agent: *`, verbatim:

```
Disallow: */search/*
Disallow: */adult/*
Disallow: /corp/*.pdf$
Disallow: /corp/solution/
Disallow: /corp/lps/
Disallow: /corp/get-data/
Disallow: /corp/unlock-growth/
Disallow: /silent-login/
Disallow: /signin-oidc/
Disallow: /signout-oidc/
```

The `/top-websites/` tree is not disallowed, and this Actor requests nothing else. Because
`*/adult/*` matches `/top-websites/adult/`, **the `adult` category has been removed from the picker**
even though Similarweb publishes it — and the Actor refuses it with an error if you send it through
the API anyway. 209 categories ship, not the 210 Similarweb lists.

***

### Pricing

**$0.001 per ranked website delivered.** 150 rows (the shipped default) = **$0.15**. The whole
worldwide taxonomy — 209 categories × 50 — is about 10,450 rows for **$10.45**.

You are charged per row you actually receive. Slices that fail to load are not billed, and a domain
repeated inside one slice is dropped before billing. `maxItems × $0.001` is a hard ceiling on what a
run can cost you.

***

### Notes and gotchas, all measured

- **A subcategory is not a subset of its parent.** `finance` and `finance/investing` are two separate
  50-row rankings. Taking both is how you go deep in a vertical, and the overlap is small.
- **`siteCategory` ≠ the slice you asked for.** A site can rank inside `finance` while its own main
  category is `finance/banking-credit-and-lending`. Both are in the row so you can group either way.
- **The same domain appears once per slice.** Ask for 3 categories and a domain that ranks in two of
  them gives you 2 rows, with different `rankingScopeCategory` values. Deduplication is per
  `country|category|domain`, which is the honest key — you are not billed twice inside one slice.
- **`dataMonth` is Similarweb's snapshot month, not today.** It rolls over once a month; every row in
  a run carries the same one.
- **Similarweb's own numbers move.** Rank, bounce rate and pages-per-visit change with each monthly
  snapshot. Schedule the Actor monthly if you want a series.

# Actor input Schema

## `categories` (type: `array`):

Pick one or more. A top-level category (Finance) and a subcategory (Finance > Investing) are separate rankings with 50 sites each, and the subcategory is NOT a subset of the parent's 50 — taking both is how you go deep in a vertical. Leave empty and tick "Include the overall ranking" to take the un-filtered top 50 instead.

## `countries` (type: `array`):

Which country's ranking to read. "Worldwide" is the global table and is the only scope that returns 50 rows together with a category. WARNING: picking a real country AND a category gives you 5 rows for that pair, not 50 — that is Similarweb's cap, not a paging limit, and the run warns you by name for every pair it affects. A country on its own (no category) returns 50.

## `includeAllCategories` (type: `boolean`):

Add the un-filtered top 50 for each country you picked — the biggest sites on the internet, or in that country, regardless of category. Costs one extra page and up to 50 extra rows per country.

## `maxItems` (type: `integer`):

Stop after this many rows. 500 = $0.50, 5,000 = $5.00. One worldwide category page yields 50 rows in about 3 seconds, so the whole 210-category worldwide taxonomy is roughly 10,500 rows in about 12 minutes.

## `proxyConfiguration` (type: `object`):

Leave as it is. This ranking route carries no bot wall — measured 24 of 24 category pages clean through Apify RESIDENTIAL — but Similarweb refuses datacenter IP ranges outright (13 of 30 datacenter calls returned a bare HTTP 403), so RESIDENTIAL is the tested default.

## Actor input object example

```json
{
  "categories": [
    "finance",
    "computers-electronics-and-technology/programming-and-developer-software",
    "e-commerce-and-shopping/marketplace"
  ],
  "countries": [
    "worldwide"
  ],
  "includeAllCategories": false,
  "maxItems": 500,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "US"
  }
}
```

# Actor output Schema

## `items` (type: `string`):

One row per ranked website per category and country slice: rank, domain, the site's own category, rank change, bounce rate, pages per visit and average visit duration.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "categories": [
        "finance",
        "computers-electronics-and-technology/programming-and-developer-software",
        "e-commerce-and-shopping/marketplace"
    ],
    "countries": [
        "worldwide"
    ],
    "maxItems": 500,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ],
        "apifyProxyCountry": "US"
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("scrapersdelight/similarweb-traffic-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "categories": [
        "finance",
        "computers-electronics-and-technology/programming-and-developer-software",
        "e-commerce-and-shopping/marketplace",
    ],
    "countries": ["worldwide"],
    "maxItems": 500,
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
        "apifyProxyCountry": "US",
    },
}

# Run the Actor and wait for it to finish
run = client.actor("scrapersdelight/similarweb-traffic-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "categories": [
    "finance",
    "computers-electronics-and-technology/programming-and-developer-software",
    "e-commerce-and-shopping/marketplace"
  ],
  "countries": [
    "worldwide"
  ],
  "maxItems": 500,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ],
    "apifyProxyCountry": "US"
  }
}' |
apify call scrapersdelight/similarweb-traffic-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,scrapersdelight/similarweb-traffic-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/zjSGSzdkYc5mF4LwZ/builds/CoeTCHEj1Szahzpq9/openapi.json