# SimilarWeb API (Scraper) (`clovys/similarweb-api`) Actor

Extract SimilarWeb website analytics: traffic, rankings, engagement, traffic sources, top keywords, and top countries. Bulk domains in parallel.

- **URL**: https://apify.com/clovys/similarweb-api.md
- **Developed by:** [Clovys](https://apify.com/clovys) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does SimilarWeb API do?

**SimilarWeb API** extracts website traffic analytics for any domain - visits, rankings, engagement, traffic sources, top countries, and keywords. Paste a list of domains (or full URLs), run the Actor, and download clean JSON / CSV / Excel from the dataset.

Built for **bulk runs** (up to **100,000** domains) with a fast default path and an automatic reliability fallback when needed. Large lists are **auto-split** into ~3,000-domain child runs on Apify, with **auto-heal** under blocks.

This Actor is **not affiliated with SimilarWeb**. Metrics are third-party estimates - use them as directional intelligence, not as audited truth.

### Key features

- **Bulk domain analysis** - up to 100,000 domains per run (invalid / duplicate inputs filtered before work starts)
- **Auto-sharding** - runs above ~3,000 domains split into child runs; one parent dataset
- **Auto-heal** - on blocks: backoff, lower concurrency, refresh sticky extension session once per wave
- **Traffic & engagement** - monthly visits, bounce rate, pages per visit, average visit duration
- **Rankings** - global rank, country rank, and category rank when available
- **Traffic sources** - direct, organic search, paid search, social, referrals, mail, display
- **Geography** - top countries with traffic share
- **Keywords** - top keywords when SimilarWeb exposes them (toggleable)
- **Success-only billing** - failed domains stay in the dataset with `error`, but are **not** charged
- **Fast default mode** - lightweight path tuned for speed and low compute (**512 MB**)
- **Automatic fallback** - optional browser path when the fast path is blocked (`fetchMode: auto`)
- **Export anywhere** - JSON, CSV, Excel, HTML from the Apify dataset

### Why Clovys

Clovys is a nonprofit organization driven by a single mission: to make the internet safer, faster, more reliable, and universally accessible.

We believe that data should not be locked behind exorbitant paywalls or complex technical barriers. By building lightweight, highly optimized Apify Actors, we help developers, researchers, and organizations access the information they need to innovate without friction.

As a nonprofit initiative, **we do not optimize for margins**. Our pricing is kept as low as possible and is strictly calculated to cover the raw infrastructure costs of data extraction - such as Apify compute units and proxy bandwidth.

For SimilarWeb specifically, Clovys focuses on delivering the most **robust, reliable, and fast** extraction path we can maintain: a production-grade default mode, retries, auto-heal, and a fallback strategy so bulk runs stay usable at scale.

### Use cases

- **Competitive analysis** - track competitors' traffic, rankings, and channel mix
- **Media & marketplace research** - score publishers, affiliates, or inventory by audience size
- **Marketing ROI** - see where traffic comes from (search, social, direct, paid) before allocating budget
- **CRM / lead enrichment** - attach traffic and rank signals to accounts
- **Investment & due diligence** - directional digital footprint for a brand or portfolio of sites
- **Content & SEO strategy** - combine visits, geo share, and top keywords for prioritization

### How to use

1. Open **SimilarWeb API** in Apify Console
2. Paste domains or URLs in **Domains** (e.g. `apify.com`, `https://www.lemonde.fr`)
3. Keep defaults for production (`fetchMode: extension`, **512 MB**)
4. Click **Start**
5. Download results from the **Dataset** tab (JSON, CSV, Excel, ...)

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `domains` | `string[]` | - | Domains or URLs to analyze (required, up to 100,000) |
| `fetchMode` | `string` | `extension` | `extension` (fast), `auto` (fast then browser), `browser`, `http` |
| `maxConcurrency` | `integer` | `5` | Parallel workers (`extension`/`http` up to 12; `browser` keep 1-3) |
| `cacheTtlHours` | `integer` | `72` | Reuse successful results within the run (`0` = off) |
| `proxyConfiguration` | `object` | - | Optional. Datacenter often enough for `extension`; use RESIDENTIAL if identity is blocked or for `auto`/`browser` |
| `includeKeywords` | `boolean` | `true` | Include top keywords when present |
| `includeCountries` | `boolean` | `true` | Include geo traffic share |
| `maxRetries` | `integer` | `1` | Retries per domain on block/extract errors |

### Output example

You can download the dataset in JSON, HTML, CSV, or Excel.

```json
{
 "domain": "apify.com",
 "title": "apify.com Traffic Analytics, Ranking & Audience",
 "globalRank": 9646,
 "visits": 4201537,
 "bounceRate": 0.37,
 "pagesPerVisit": 7.43,
 "avgVisitDuration": 300,
 "trafficSources": {
 "direct": 0.44,
 "searchOrganic": 0.35
 },
 "topCountries": [
 { "country": "United States", "value": 0.21 }
 ],
 "scrapedAt": "2026-09-24T12:00:00.000Z",
 "error": null
}
```

Failed domains still appear with `error` set and metrics `null` - **they are not billed**.

### Data fields

| Field | Description |
|---|---|
| `domain` | Normalized hostname |
| `title` | Site title / SimilarWeb label |
| `globalRank` | Worldwide traffic rank |
| `visits` | Estimated monthly visits |
| `bounceRate` | Bounce rate |
| `pagesPerVisit` | Pages per visit |
| `avgVisitDuration` | Average visit duration (seconds) |
| `trafficSources` | Channel mix (direct, search, social, ...) |
| `topCountries` | Top geos with share |
| `topKeywords` | Top keywords when enabled / available |
| `error` | Error message if the domain failed |

### Pricing

Pay-per-event pricing (platform usage is billed separately by Apify):

| Event | When charged | Price |
|---|---|---|
| **result** | Each successful domain written to the dataset | **$0.005** (**$5 / 1,000**) |
| **Actor Start** | Once when the run starts (scaled by memory GB) | **$0.001** |

Example: **100 successful domains** ~ **$0.50** in PPE `result` (+ Actor Start + Apify platform usage).

Only successful results are charged as `result`. Failed domains are free on the PPE event.

### Recommended settings

| Goal | Settings |
|---|---|
| Production / Store | `fetchMode: extension`, **512 MB**, concurrency **5-10** |
| Max reliability | `fetchMode: auto`, RESIDENTIAL proxy if needed, **2048 MB** |
| Browser-only | `fetchMode: browser`, concurrency **1-2**, **2048 MB** |
| Huge lists | up to **100,000** domains - auto-shard handles the rest |

### FAQ

**How many domains per run?** Up to **100,000**. Lists above ~3,000 are auto-split into child runs; invalid / duplicate inputs are dropped **before** sharding. Parallelism is controlled by `maxConcurrency`.

**What if a domain fails?** It is stored with `error`. Other domains continue. Failed items are **not** billed as `result`.

**Do I need expensive proxies?** Often **no** for `fetchMode: extension` with Apify datacenter / automatic proxy. If extension identity returns 403, switch to **RESIDENTIAL** and/or `fetchMode: auto`.

**What does auto-heal do?** On block waves it slows down, lowers concurrency, and refreshes the sticky extension session once per wave - so the run keeps going instead of burning proxy on blind retries.

**Is this official SimilarWeb data?** No. This Actor is independent and unaffiliated. Numbers are estimates from public SimilarWeb surfaces.

### Support

Questions, bugs, or feature requests: use the Actor **Issues** tab on Apify, or contact Clovys through the Store profile.

# Actor input Schema

## `domains` (type: `array`):

List of domains or URLs to analyze (e.g. apify.com, https://www.google.com). Up to 100 000 per run (auto-split into ~3k chunks on Apify).

## `fetchMode` (type: `string`):

extension = fast lightweight API (recommended). auto = extension then browser fallback. browser = Camoufox only. http = legacy Impit website fetch.

## `maxConcurrency` (type: `integer`):

Parallel workers. Fast modes: 5–10. Browser mode: keep 1–3. Auto-heal may temporarily lower this under blocks.

## `proxyConfiguration` (type: `object`):

Datacenter / Apify automatic proxy is enough for the default fast mode. Residential is optional for browser fallback.

## `cacheTtlHours` (type: `integer`):

Reuse successful results already stored in this run's KV store. 0 disables. Default 72h saves cost on repeated domains.

## `includeKeywords` (type: `boolean`):

Include top organic keywords when available.

## `includeCountries` (type: `boolean`):

Include geographic traffic share breakdown.

## `maxRetries` (type: `integer`):

Retries on network / block errors.

## `extensionVersion` (type: `string`):

Advanced: only change if support instructs you to. Used by the fast fetch mode.

## `bootstrapCookies` (type: `boolean`):

Advanced: seed cookies via browser for legacy http mode. Leave off.

## `_shard` (type: `boolean`):

Internal self-shard marker. Do not set manually.

## `_parentDatasetId` (type: `string`):

Internal parent dataset id for self-shard children. Do not set manually.

## Actor input object example

```json
{
  "domains": [
    "apify.com"
  ],
  "fetchMode": "extension",
  "maxConcurrency": 5,
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "cacheTtlHours": 72,
  "includeKeywords": true,
  "includeCountries": true,
  "maxRetries": 1,
  "extensionVersion": "6.12.18",
  "bootstrapCookies": false,
  "_shard": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "apify.com"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("clovys/similarweb-api").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["apify.com"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("clovys/similarweb-api").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "apify.com"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call clovys/similarweb-api --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,clovys/similarweb-api"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wpVh8K9zpp9DBg9Rj/builds/e5tSVt6dpjykjFAAK/openapi.json