# Similarweb Scraper — Website Traffic & Competitors (`apivault_labs/similarweb-scraper`) Actor

Analyze up to 10,000 domains: monthly visits, rankings, engagement, traffic channels, countries, keywords, competitors, AI referrals, domain registration and technology insights. Compare sites, monitor changes, map markets and export structured data. No external account or API credentials required.

- **URL**: https://apify.com/apivault\_labs/similarweb-scraper.md
- **Developed by:** [Apivault Labs](https://apify.com/apivault_labs) (community)
- **Categories:** SEO tools, Lead generation, Automation
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 website traffic analysis results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Similarweb Scraper — Website Traffic, AI Traffic & Competitor Analysis

Fast Similarweb scraper and website traffic API alternative for competitive intelligence, SEO research and market analysis. Enter one website or up to **10,000 domains** to get monthly visits, rankings, engagement, traffic sources, top countries, keywords, competitors and AI/LLM referral insights.

No external analytics account or API credentials are required. Export clean, structured results to JSON, CSV or Excel, or connect the Actor to n8n, Make, Zapier, dashboards and data warehouses.

### Why choose this Similarweb scraper?

- **Mega-fast bulk website traffic analysis** — lightweight parallel processing for up to 10,000 domains.
- **Deeper AI traffic intelligence** — AI referral share, estimated visits, source ranking, three-month source trends and available audience prompts.
- **All-in-one competitor research** — traffic analytics, similar websites, domain registration insights, technology signals, social profiles and keyword insights.
- **Seven workflows in one Actor** — analyze, compare, monitor, map markets and discover top or trending websites.
- **Predictable bulk output** — one clearly marked result per valid domain, stable ordering and retry-ready failures.
- **Automation-ready website traffic API** — structured fields and focused dataset views for production workflows.

### Seven modes

#### 1. Traffic Overview

The fastest mode for lists of up to **10,000 domains**:

- monthly visits and three-month history;
- month-over-month and period growth;
- global, country and category ranks;
- bounce rate, pages per visit and visit duration;
- complete acquisition-channel split;
- organic, paid and AI visit estimates;
- top countries and organic keywords;
- AI/LLM referral share, source ranking and chatbot distribution;
- three-month AI-source trends and available top audience prompts.

#### 2. Similar Sites

Find up to 20 competitors and alternatives for each domain, including:

- similarity rank and score;
- estimated monthly visits;
- category and country rank;
- description and preview URL.

#### 3. Complete Profile

Combines traffic and competitors with expanded domain and company intelligence:

- registration date, domain age, expiry, registrar and nameservers;
- DNSSEC and domain status;
- company name, logo and favicon;
- public social profiles;
- detected CMS, e-commerce platform, frameworks, analytics and marketing tools;
- website keyword insights and visible content metrics.

#### 4. Compare Websites

The first input domain becomes the primary website. Every output row receives:

- traffic, growth and AI rank within the batch;
- share of combined traffic;
- traffic gap and relative size versus the primary website;
- acquisition-channel gaps;
- leader, challenger, mid-pack or niche positioning.

A compact cohort report is saved separately as `COMPARISON` without creating an additional dataset row.

#### 5. Monitor Changes

Use the same `monitorKey` in recurring or scheduled runs. The Actor remembers the previous baseline and reports:

- traffic and rank changes;
- acquisition-channel shifts;
- added or removed keywords;
- added or removed competitors;
- technology changes in Complete Profile monitoring.

Choose low, medium or high sensitivity. Enable `onlyChanges` to keep unchanged domains out of the dataset.

#### 6. Market Map

Enter one or more seed websites. The Actor discovers their strongest competitors, enriches the selected domains and returns:

- market-map nodes and seed relationships;
- connection count per website;
- mapped traffic share;
- traffic, growth, rank, category and AI metrics;
- graph edges in the `MARKET_MAP` output record.

#### 7. Website Discovery

No domain list is required. Discover:

- top websites worldwide or within a country;
- category leaders;
- rising and falling websites;
- new top-list entries and recently dropped websites.

Enable enrichment to add traffic, growth, channels, keywords and AI metrics to every discovered domain.

### Quick start

#### Fast traffic check

```json
{
  "mode": "overview",
  "domains": ["apify.com", "shopify.com", "openai.com"]
}
```

#### Complete competitor profiles

```json
{
  "mode": "all_in_one",
  "domains": ["apify.com", "scraperapi.com"],
  "maxConcurrency": 3
}
```

#### Side-by-side comparison

```json
{
  "mode": "compare",
  "domains": ["apify.com", "scraperapi.com", "brightdata.com"]
}
```

#### Weekly monitoring

```json
{
  "mode": "monitor",
  "domains": ["apify.com", "scraperapi.com"],
  "monitorKey": "scraping-platforms",
  "changeThreshold": "medium",
  "onlyChanges": true
}
```

#### Competitive market map

```json
{
  "mode": "market_map",
  "domains": ["apify.com"],
  "marketMapMaxCompetitors": 20
}
```

#### Top e-commerce websites in the United States

```json
{
  "mode": "discovery",
  "discoveryType": "top",
  "discoveryCountry": "united-states",
  "discoveryCategory": "e-commerce-and-shopping",
  "enrichDiscovery": true
}
```

#### Trending websites

```json
{
  "mode": "discovery",
  "discoveryType": "trending",
  "discoveryCountry": "united-states",
  "enrichDiscovery": true
}
```

### Outputs

- **Dataset** — one structured row per returned website. By default, every unique valid input domain receives either data or a clearly marked status row.
- **OUTPUT** — run statistics and mode-specific summary.
- **FAILED\_DOMAINS** — invalid, unavailable or zero-data domains ready for retry.
- **COMPARISON** — cohort rankings and leaders.
- **MONITOR\_SUMMARY** — baseline and changed-domain counts.
- **MARKET\_MAP** — graph edges and mapped-market summary.
- **DISCOVERY\_SUMMARY** — discovery settings and result count.

Focused dataset views make Overview, Acquisition & AI, Competitors, Complete Profile, Comparison, Monitoring, Market Map and Discovery easy to inspect in the Apify Console.

### Reliability and scale

- Automatic URL normalization and deduplication.
- Stable output order for bulk input reconciliation.
- Explicit `success`, `no_data` or `failed` processing status.
- Field-level coverage flags instead of ambiguous empty values.
- Failed-domain output for partial reruns.
- Lightweight Overview mode for high-volume enrichment.
- Explicit limits for market-map expansion.
- Run summaries include duplicate count, status totals and data-completeness percentage.

Set `returnFailureRecords` to `false` if you prefer successful dataset rows only. Retry-ready failures remain available separately in `FAILED_DOMAINS`.

Availability varies by website. Traffic figures are estimates rather than first-party analytics. Smaller or recently launched domains may have limited data.

### FAQ

**Do I need to connect an external analytics account?**\
No.

**How many domains can Overview process?**\
Up to 10,000 unique domains per run.

**Are competitors available for every website?**\
Coverage is broad, but some new or very small domains may have no related-site data.

**Does monitoring work with Apify schedules?**\
Yes. Schedule the same input and keep the same `monitorKey`.

**Is this an official Similarweb product?**\
No. This is an independent Actor and is not affiliated with or endorsed by Similarweb. Similarweb is a trademark of its respective owner.

# Actor input Schema

## `mode` (type: `string`):

Choose a focused workflow, from a fast traffic overview to competitor discovery, monitoring and market mapping.

## `domains` (type: `array`):

Required for every mode except Discovery. Up to 10,000 domains; duplicates are removed automatically. In Compare, the first domain is the primary website. In Market Map, these are seed websites.

## `includeSimilarSites` (type: `boolean`):

Adds a reliable competitor lookup to every overview record. Off by default to preserve maximum bulk speed. Similar Sites, Complete Profile and Market Map modes enable it automatically.

## `includeTopKeywords` (type: `boolean`):

Include top search keywords and available volume, CPC and estimated traffic.

## `includeAiTraffic` (type: `boolean`):

Include generative-AI traffic share, estimated visits, source ranking, three-month source trends and available top audience prompts.

## `includeScreenshot` (type: `boolean`):

Include the available website preview image URL.

## `includeRawData` (type: `boolean`):

Adds a nested raw\_data field. Best for advanced analysis and small runs.

## `includeNoData` (type: `boolean`):

Keep one clearly marked dataset row for valid domains even when traffic coverage is unavailable. Recommended for reliable bulk reconciliation.

## `returnFailureRecords` (type: `boolean`):

Keep a status row with a retry reason when a valid domain cannot be fetched, so bulk outputs do not silently lose inputs.

## `monitorKey` (type: `string`):

Use the same name on future scheduled runs to compare against the saved baseline.

## `monitorDepth` (type: `string`):

Choose a fast traffic baseline or a complete website intelligence baseline.

## `changeThreshold` (type: `string`):

Controls how large a movement must be before it is reported as a material change.

## `onlyChanges` (type: `boolean`):

After the baseline run, omit unchanged domains from the dataset.

## `marketMapMaxCompetitors` (type: `integer`):

Maximum number of unique competitors selected across all seed domains, ranked by similarity.

## `discoveryType` (type: `string`):

Find either leading websites by rank or websites currently gaining, losing, entering or leaving visibility.

## `discoveryCountry` (type: `string`):

Country URL name, for example worldwide, united-states, united-kingdom, germany, india or brazil.

## `discoveryCategory` (type: `string`):

For example e-commerce-and-shopping, finance, health, games or travel-and-tourism.

## `maxResults` (type: `integer`):

Public ranking pages normally contain up to 50 sites. Trending lists contain rising, falling, new and dropped groups.

## `enrichDiscovery` (type: `boolean`):

Enrich every discovered domain with visits, ranks, channels, keywords and AI traffic.

## `maxConcurrency` (type: `integer`):

Overview supports higher concurrency. Use 3–5 for Complete Profile and Market Map.

## `timeout` (type: `integer`):

Maximum time allowed for each external request before retrying or moving on.

## `maxRetries` (type: `integer`):

Automatic retries for temporary blocks, rate limits and server errors.

## Actor input object example

```json
{
  "mode": "overview",
  "domains": [
    "apify.com",
    "scraperapi.com",
    "brightdata.com"
  ],
  "includeSimilarSites": false,
  "includeTopKeywords": true,
  "includeAiTraffic": true,
  "includeScreenshot": false,
  "includeRawData": false,
  "includeNoData": true,
  "returnFailureRecords": true,
  "monitorKey": "my-competitors",
  "monitorDepth": "overview",
  "changeThreshold": "medium",
  "onlyChanges": false,
  "marketMapMaxCompetitors": 20,
  "discoveryType": "top",
  "discoveryCountry": "worldwide",
  "maxResults": 50,
  "enrichDiscovery": true,
  "maxConcurrency": 10,
  "timeout": 15,
  "maxRetries": 2
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `failedDomains` (type: `string`):

No description

## `comparison` (type: `string`):

No description

## `monitorSummary` (type: `string`):

No description

## `marketMap` (type: `string`):

No description

## `discoverySummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "apify.com",
        "scraperapi.com",
        "brightdata.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("apivault_labs/similarweb-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "apify.com",
        "scraperapi.com",
        "brightdata.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("apivault_labs/similarweb-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "apify.com",
    "scraperapi.com",
    "brightdata.com"
  ]
}' |
apify call apivault_labs/similarweb-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,apivault_labs/similarweb-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/STGjEsbNydv95Thzt/builds/6bvN7H21X9OffAcSH/openapi.json