# Similarweb Scraper: Website Traffic & Rank, No Login (`themineworks/similarweb-scraper`) Actor

Scrape Similarweb's free public website overview: estimated monthly visits, engagement, traffic-source split, top countries & similar sites. No login, no paid Similarweb seat. Use it as an MCP server in Claude, ChatGPT & AI agents.

- **URL**: https://apify.com/themineworks/similarweb-scraper.md
- **Developed by:** [The Mine Works](https://apify.com/themineworks) (community)
- **Categories:** SEO tools, Developer tools, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.40 / 1,000 domain scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Similarweb Scraper: Website Traffic & Rank, No Login

> **The Mine Works:** 544 users across 100 public Actors, >99% run success. Pay only for results delivered. [Browse all Actors](https://apify.com/themineworks).

### Overview

Similarweb Scraper turns any domain into a structured JSON record of what Similarweb's **free, public, logged-out website-overview tool** actually shows: estimated global rank, category, estimated monthly visits and month-over-month trend, engagement (bounce rate, pages per visit, average visit duration), a traffic-source split (direct / search / social / referrals / paid search / mail / display ads), the top countries by traffic share, and the similar/competitor sites Similarweb lists for that domain.

It is built for people who want a quick, no-account read on a domain's traffic profile: growth marketers sizing up a competitor, SEOs doing a first-pass audit, sales teams qualifying a prospect's web presence, and researchers building a traffic-comparison dataset across many domains. No Similarweb Pro seat, no Data API contract, no manual copy-paste from the browser.

✅ No login required | ✅ No API key | ✅ Fast — no browser required | ✅ MCP-ready for AI agents

### Features

Bulk domain lookup — pass up to 50 domains in one run.
Global rank, category, category rank, and country rank, when Similarweb shows them.
Estimated monthly visits with the month-over-month change shown on the free overview.
Engagement metrics: bounce rate, pages per visit, average visit duration.
Traffic-source split across the channels Similarweb's free tool breaks out.
Top countries by traffic share, and the similar-sites list Similarweb surfaces.
Pay per domain actually delivered — blocked or empty lookups are never charged.

### How it works

The actor fetches `similarweb.com/website/<domain>` directly with plain HTTP through Apify's UNBLOCKER proxy group — no browser, no on-page search simulation. UNBLOCKER solves whatever's in front of the page server-side and hands back a real, fully-rendered HTML response; the report itself is embedded in the page as a `window.__APP_DATA__` JSON blob, which the actor parses directly for clean, structured fields (rank, category, visits, traffic sources, top countries, similar sites) instead of scraping rendered text off the DOM. A full report typically takes 30-70 seconds to resolve server-side, so the actor retries a domain with a fresh proxy session (`maxRetriesPerDomain`) when a request comes back mid-solve rather than fully rendered — most domains land on the first or second attempt.

### 🧾 Input configuration

```json
{
  "domains": ["wikipedia.org"],
  "maxRetriesPerDomain": 3
}
```

| Field | Type | Description |
| --- | --- | --- |
| `domains` | array of strings | Up to 50 bare root domains (no protocol, no path) — e.g. `"wikipedia.org"`. Required. |
| `maxRetriesPerDomain` | integer | How many times to retry a domain with a fresh residential IP session if Similarweb blocks the request (default `3`, max `6`). |

### 📤 Output format

```json
{
  "domain": "wikipedia.org",
  "url": "https://www.similarweb.com/website/wikipedia.org",
  "global_rank": 7,
  "category": "Encyclopedias and Reference",
  "total_visits": "4.2B",
  "visits_change_pct": "-1.3%",
  "bounce_rate_pct": 41.2,
  "pages_per_visit": 2.9,
  "avg_visit_duration": "00:04:12",
  "traffic_sources": { "direct": 55.1, "search": 38.4, "social": 1.2, "referrals": 4.1 },
  "top_countries": [{ "country": "United States", "share_pct": 18.4 }],
  "similar_sites": ["britannica.com", "quora.com"],
  "checked_at": "2026-08-02T15:30:00.000Z"
}
```

Every field is only ever set when Similarweb's overview page actually showed it — fields it does not display for a given domain are omitted rather than returned as `null` or a guessed value, so downstream parsing stays clean and honest.

### 💼 Common use cases

**Competitive traffic benchmarking**
Pull estimated visits, rank, and traffic-source mix for a list of competitors in one run, refreshed on a schedule.

**Prospect and lead qualification**
Check a prospect's web traffic scale and channel mix before a sales call, without a Similarweb seat.

**SEO and content audits**
Start an audit with a domain's category, rank, and top traffic-driving channel before going deeper with dedicated SEO tools.

**Market and competitor mapping**
Feed a list of domains through the actor to build a comparison table of traffic scale and top countries across a whole category.

### 🚀 Getting started

1. Open the actor and list the domains you want to check under `domains` (bare domain, e.g. `nike.com`).
2. Leave `maxRetriesPerDomain` at the default unless you want more or fewer retries per domain.
3. Click Start. The actor fetches each domain's Similarweb overview page directly and parses the embedded report data.
4. Download results as JSON, CSV, or Excel, or pull them via the Apify API.

### FAQ

**Do I need a Similarweb account or API key?**
No. The actor reads the exact same free, public overview a logged-out visitor gets at similarweb.com. You never log in and never buy a Similarweb Pro seat or Data API plan.

**How is pricing calculated?**
Pay per event. You are charged only for each domain where real overview data was actually extracted. Blocked, rate-limited, or empty lookups are never charged. There is no free tier.

**Does this need a browser or residential proxy?**
No. The actor fetches the report page with plain HTTP through Apify's UNBLOCKER proxy group, which solves whatever anti-bot layer sits in front of the page server-side and returns a fully-rendered response — no browser process on the actor's side at all.

**Does this get the same data as Similarweb Pro?**
No, and it never will. This actor is scoped to exactly what the free, logged-out overview tool shows. Similarweb's keyword-level reports, competitor benchmarking suite, and Data API are paid products that sit behind a login; this actor does not attempt to access them.

**What formats can I export?**
Results are stored in an Apify dataset and can be exported as JSON, CSV, Excel, or XML, or pulled programmatically via the Apify API.

### Coverage & known limitations (read this before you rely on it)

This actor is scoped, on purpose, to Similarweb's free public overview only — not the paid Pro platform, not the Data API, not keyword-level or competitor-benchmarking reports. Within that scope, be aware of two real constraints we verified directly while building this:

1. **A full report takes real time to resolve.** UNBLOCKER renders the page server-side, and a genuine domain report was observed taking anywhere from ~30s to ~70s to fully land; a request that comes back truncated or as a transient 202/502 mid-solve is retried with a fresh proxy session rather than treated as a failure. Most domains land within the first two attempts.
2. **Some domains genuinely have no Similarweb report.** Very low-traffic or unrecognized domains return the tool's generic landing page instead of a report — this is a real, honest "no data" outcome (visible in the dataset as a `no_data` row and in the run summary's `no_data_count`), not a block, and it is never charged.

If Similarweb changes the page's markup or embedded-data shape, expect a build update rather than fabricated results — the actor never invents a field it did not actually find in the response.

### Use in Claude, ChatGPT & any MCP agent

```
https://mcp.apify.com/?tools=themineworks/similarweb-scraper
```

Or call it from code with the Apify client:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/similarweb-scraper').call({
  domains: ['wikipedia.org', 'nike.com'],
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### 🛠️ Complete your competitive-intelligence pipeline

Pair this scraper with the rest of the themineworks competitive-data suite:

- **[Crunchbase Scraper](https://apify.com/themineworks/crunchbase-companies)**: company funding, investors, and firmographics.
- **[Google Trends Scraper](https://apify.com/themineworks/google-trends-pro)**: search interest and rising queries alongside traffic data.
- **[Company Identity Resolver](https://apify.com/themineworks/company-identity-resolver)**: verify companies via LEI, EU registration, and SEC EDGAR.

Typical flow: similarweb-scraper pulls traffic scale and channel mix, crunchbase-companies layers on funding stage, and google-trends-pro adds search-demand trend on top.

Found a bug or have a feature request? Open an issue on the actor's Apify Console page or reach out through the Apify profile.

*Last verified: 2026-08*

# Actor input Schema

## `domains` (type: `array`):

One or more root domains to look up on Similarweb's free public overview tool (e.g. "wikipedia.org"). No protocol, no path — bare domain only. Each domain is one billed result.

## `maxRetriesPerDomain` (type: `integer`):

How many times to retry a domain with a fresh proxy session if the report doesn't fully load. A full report can take 30-70s to render server-side, so retrying with a new session materially helps.

## Actor input object example

```json
{
  "domains": [
    "wikipedia.org"
  ],
  "maxRetriesPerDomain": 3
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "wikipedia.org"
    ],
    "maxRetriesPerDomain": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("themineworks/similarweb-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["wikipedia.org"],
    "maxRetriesPerDomain": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("themineworks/similarweb-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "wikipedia.org"
  ],
  "maxRetriesPerDomain": 3
}' |
apify call themineworks/similarweb-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,themineworks/similarweb-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ExJiZcZRD0COSF4d5/builds/k9nerTAX04O14FpY4/openapi.json