Similarweb Traffic Intelligence Scraper

Turn a list of websites into structured traffic-intelligence records.

This Actor accepts domains or full URLs and extracts public Similarweb estimates for visits, rankings, engagement, acquisition channels, countries, and search keywords. It is designed for repeatable competitor monitoring, SEO research, agency reporting, investment diligence, and lead scoring.

No Similarweb account or API key is required.

What does Similarweb Traffic Intelligence Scraper do?

The Actor performs one lookup per unique domain and saves one normalized dataset row.

It can:

• 🌐 normalize bare domains and full URLs;
• 📈 extract current and monthly estimated visits;
• 🏆 collect global, country, and category ranks;
• ⏱️ return bounce rate, pages per visit, and time on site;
• 🚦 break traffic down by direct, search, social, referral, mail, ads, affiliate, and generative AI;
• 🌍 list leading countries and their traffic shares;
• 🔎 return top keywords with volume and CPC when available;
• 📦 process a bulk domain cohort with bounded concurrency;
• 🔁 retry blocked requests with fresh sessions;
• 💾 export results as JSON, CSV, Excel, XML, RSS, or via API.

Who is it for?

SEO and growth teams

Compare organic visibility, paid-search share, engagement, and traffic direction across competitors.

Competitive-intelligence analysts

Refresh a watchlist on a weekly or monthly schedule and join snapshots by domain.

Agencies

Build repeatable client and prospect reports without manually copying website analytics.

Investors and researchers

Use estimated visits, market ranks, countries, and channel mix as directional diligence signals.

Sales and lead-scoring teams

Enrich domain lists with traffic scale and engagement indicators before prioritizing outreach.

Why use this Actor?

Similarweb's normal website pages are protected and tedious to inspect at scale. This Actor uses a lightweight public data surface with managed proxy sessions and produces a stable schema.

You get:

• one row per domain;
• source snapshot dates separated from scrape timestamps;
• explicit metric names and units;
• typed arrays for months, countries, and keywords;
• duplicate input removal;
• no charge for empty or blocked records;
• non-zero failure when every lookup fails.

What data can you extract?

Field	Type	Description
domain	string	Canonical domain without protocol or www
similarwebUrl	URL	Public Similarweb website page
snapshotDate	datetime	Date represented by the source metrics
scrapedAt	datetime	Time the Actor retrieved the record
title	string	Website title
description	string	Website description
category	string	Similarweb category
primaryCountryCode	string	Main country ISO code
globalRank	number	Worldwide traffic rank
countryRank	number	Rank in the main country
categoryRank	number	Rank in the category
totalVisits	number	Latest monthly estimated visits
monthlyVisits	array	Recent month and visit values
bounceRate	number	Estimated single-page visit share
pagesPerVisit	number	Estimated pages per session
timeOnSiteSeconds	number	Estimated average visit duration
trafficChannels	object	Acquisition-channel shares
topCountries	array	Country names, codes, and shares
topKeywords	array	Keyword, traffic value, volume, and CPC
isSmallSite	boolean	Source low-traffic coverage marker

How to scrape Similarweb traffic data

1. Open the Actor input page.
2. Add one or more domains or URLs.
3. Leave parallel lookups at 3 for reliable bulk processing.
4. Keep the recommended residential proxy configuration.
5. Click Start.
6. Open the Dataset tab when the run finishes.
7. Export the data or consume it through the API.

A small first run with two domains normally completes well under two minutes.

Input

{
  "domains": [
    "apify.com",
    "https://www.openai.com/research/"
  ],
  "maxConcurrency": 2,
  "maxRetries": 2
}

domains

Required array with up to 1,000 entries. Accepted forms include:

• apify.com
• www.apify.com
• https://apify.com/
• https://apify.com/store

All four resolve to the canonical domain apify.com. Duplicate domains are removed before requests begin.

maxConcurrency

Number of parallel domain lookups, from 1 to 10. The default is 3. Higher values can increase blocking and are not always faster.

maxRetries

Number of fresh-session retries per domain, from 0 to 5. The default is 2.

proxyConfiguration

The default uses US residential Apify Proxy because the source blocks ordinary datacenter requests. You can provide your own compatible proxies.

Output example

{
  "domain": "apify.com",
  "similarwebUrl": "https://www.similarweb.com/website/apify.com/",
  "snapshotDate": "2026-06-01T00:00:00+00:00",
  "scrapedAt": "2026-07-17T17:00:00.000Z",
  "title": "Apify: Full-stack web scraping and data extraction platform",
  "category": "Computers_Electronics_and_Technology/Computers_Electronics_and_Technology",
  "primaryCountryCode": "US",
  "globalRank": 8801,
  "countryRank": 8256,
  "categoryRank": 247,
  "totalVisits": 4414475,
  "monthlyVisits": [
    { "month": "2026-04-01", "visits": 4130336 },
    { "month": "2026-05-01", "visits": 4418959 },
    { "month": "2026-06-01", "visits": 4414475 }
  ],
  "bounceRate": 0.3635451162,
  "pagesPerVisit": 7.3124135947,
  "timeOnSiteSeconds": 294.1571081669,
  "trafficChannels": {
    "direct": 0.4145422251,
    "organicSearch": 0.366193478,
    "paidSearch": 0.0575787237,
    "generativeAi": 0.0309050553
  },
  "topCountries": [
    { "countryCode": "US", "countryName": "United States", "share": 0.2108480084 }
  ],
  "topKeywords": [
    { "keyword": "apify", "estimatedTrafficValue": 1077780, "volume": 953570, "cpc": 0.83 }
  ],
  "isSmallSite": false
}

Values are estimates supplied by the source. They should not be interpreted as first-party analytics.

How much does it cost to scrape Similarweb traffic data?

The Actor uses pay-per-event pricing:

• the run-start event is $0.005 and covers session initialization;
• one result event is charged only after a useful domain record is saved;
• blocked, empty, or invalid records are not charged as results;
• result prices are $0.0030811 FREE, $0.0026792 BRONZE, $0.0020898 SILVER, $0.0016075 GOLD, $0.0010717 PLATINUM, and $0.00075018 DIAMOND.

The Apify Console displays the exact current price before each run. Proxy and compute usage is included in platform execution accounting.

Traffic-channel interpretation

Channel values are shares between 0 and 1.

For example, organicSearch: 0.36 means roughly 36% of estimated visits came from organic search in the source model.

Possible keys include:

• direct
• organicSearch
• paidSearch
• organicSocial
• paidSocial
• referrals
• mail
• displayAds
• affiliate
• generativeAi

A missing key means the source did not provide that metric for the domain.

Monthly monitoring workflow

Use an Apify schedule to run the same domain cohort every month.

Recommended workflow:

1. create one task with your stable domain list;
2. schedule it after the source's monthly refresh;
3. send the dataset to Google Sheets, a warehouse, or a webhook;
4. retain snapshotDate and scrapedAt;
5. compare visits, rank, engagement, and channel share over time.

Do not overwrite historical rows if trend analysis matters.

Lead-scoring workflow

Traffic intelligence can enrich a CRM or prospect dataset.

Example scoring signals:

• estimated visits above a chosen threshold;
• improving global rank;
• high paid-search or display-ad share;
• target-country concentration;
• meaningful brand keyword volume;
• engagement above a segment benchmark.

Treat these as directional indicators, not verified business revenue.

SEO competitor workflow

Build a list of direct competitors and compare:

• monthly visits;
• organic versus paid search share;
• top search keywords;
• bounce rate and pages per visit;
• category rank;
• leading countries.

The normalized output makes it straightforward to calculate deltas in a spreadsheet or BI tool.

Integrations

Google Sheets

Use the Google Sheets integration to append each scheduled snapshot to a reporting workbook.

Webhooks

Send a run-finished webhook to Slack, Make, Zapier, n8n, or an internal ingestion endpoint.

Cloud storage and warehouses

Export JSON or CSV to S3-compatible storage, BigQuery, Snowflake, or your data lake pipeline.

Other Apify Actors

Pass a domain list from a lead scraper into this Actor, then join records by normalized domain.

JavaScript API example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/similarweb-traffic-intelligence-scraper').call({
  domains: ['apify.com', 'openai.com'],
  maxConcurrency: 2,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python API example

from apify_client import ApifyClient
import os

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('automation-lab/similarweb-traffic-intelligence-scraper').call(
    run_input={'domains': ['apify.com', 'openai.com'], 'maxConcurrency': 2}
)
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL API example

curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~similarweb-traffic-intelligence-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["apify.com","openai.com"],"maxConcurrency":2}'

Fetch the default dataset after the run succeeds.

Use with Apify MCP

Connect the Actor to Claude and other MCP clients through:

https://mcp.apify.com/?tools=automation-lab/similarweb-traffic-intelligence-scraper

Claude Code setup

$claude mcp add --transport http apify "https://mcp.apify.com/?tools=automation-lab/similarweb-traffic-intelligence-scraper"

Example prompt:

Run the Similarweb traffic scraper for apify.com, openai.com, and anthropic.com. Compare visits, engagement, and organic search share.

Claude Desktop setup

Add this to the Claude Desktop MCP configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=automation-lab/similarweb-traffic-intelligence-scraper"
    }
  }
}

Example prompt:

Refresh my SaaS competitor domain cohort and summarize the largest month-over-month traffic changes.

Cursor setup

Add the same mcpServers JSON to Cursor's MCP settings, then ask:

Compare traffic channels and top countries for these five competitor domains.

VS Code setup

Add an HTTP MCP server using the URL above in VS Code's MCP configuration. Then ask:

Extract Similarweb traffic intelligence for my selected domains and save the result as a comparison table.

MCP is useful when an analyst wants both the raw dataset and a narrative interpretation.

Tips for reliable runs

• Keep concurrency at 3 unless you have tested a larger cohort.
• Use residential proxies; normal datacenter IPs are commonly blocked.
• Keep retries enabled for scheduled monitoring.
• Submit canonical domains when possible.
• Split very large watchlists into scheduled batches.
• Review isSmallSite and missing metrics before making comparisons.
• Use the source snapshot date rather than the run date for monthly charts.

Error handling

Invalid domains fail validation before scraping.

For mixed batches:

• successful domains are saved;
• failed domains are logged;
• only saved records are charged;
• the run succeeds if at least one useful record is produced.

If every domain is blocked, empty, or unavailable, the run fails non-zero. This prevents silent empty monitoring runs.

Data quality and limitations

Similarweb values are modeled estimates, not a website owner's Google Analytics or server logs.

Coverage can vary for:

• new domains;
• low-traffic websites;
• subdomains;
• recently redirected domains;
• sites with limited regional signals.

Metrics may be revised by the source. Compare trends using the same methodology and avoid presenting estimates as audited facts.

Is scraping Similarweb legal?

This Actor accesses publicly available aggregate website metrics and does not collect private user data.

Scraping legality depends on your location, purpose, contract, and downstream use. You are responsible for complying with applicable laws, the source terms, privacy requirements, and intellectual-property rules.

Use reasonable request volume. Do not use output for unlawful discrimination, deception, or privacy-invasive profiling.

FAQ

Do I need a Similarweb account or API key?

No. The Actor uses a public data surface and managed proxy sessions.

Can I submit full URLs?

Yes. Protocols, paths, query strings, and www prefixes are normalized to a domain.

Why is a domain missing metrics?

The source may have insufficient traffic coverage, or a request may have been blocked. Check logs and the isSmallSite field. Scheduled retries often help transient blocks.

Why did the whole run fail?

The Actor intentionally fails when no domain produces useful data. Verify proxy configuration, domain spelling, and source availability.

Are the numbers exact?

No. They are third-party estimates intended for directional comparison.

Can I process subdomains?

Yes, but coverage depends on whether Similarweb models that subdomain separately.

Does the Actor return demographics or technology stacks?

Not in v1. The product focuses on the proven base traffic-intelligence contract.

How often should I refresh a cohort?

Monthly is appropriate for most traffic trends. Weekly runs can help monitor rank or cohort changes, but the source's visit history is typically monthly.

Related scrapers

Combine this Actor with tools from automation-lab:

• Website Change Monitor for content and page-change alerts.
• Sitemap URL Status Auditor for crawlability and URL health.
• B2B URL Finder for discovering company websites before traffic enrichment.

Support

If a run fails on valid domains, open an Actor issue and include:

• the run URL;
• one affected domain;
• whether the failure is intermittent;
• the approximate time of the run.

Do not post API tokens, proxy passwords, or private datasets.

Start your first traffic-intelligence run

Add two competitor domains, keep the default reliability settings, and start the Actor. The resulting dataset is ready for export, scheduling, API use, and downstream analysis.