Similarweb Traffic Scraper

Scrape SimilarWeb website traffic estimates at scale. Extract monthly visits, bounce rates, traffic source breakdowns, geographic distribution, top referral sites, and competitor analysis for any domain.

Input Parameters

Field	Type	Required	Default	Description
domains	array of strings	Yes	-	List of domains to analyze. Enter bare domains (e.g., "google.com") or full SimilarWeb URLs.
includeCompetitors	boolean	No	true	Extract competitor/similar sites data for each domain. Doubles page loads per domain.
includeTrafficSources	boolean	No	true	Extract detailed traffic source percentages (direct, referral, search, social, mail, display).
includeGeography	boolean	No	true	Extract top countries by traffic share.
includeReferrals	boolean	No	true	Extract top referring sites and top destination sites.
maxResults	integer	No	50	Maximum number of domains to analyze. Range: 1-1000.
proxyConfiguration	object	No	Residential	Proxy settings. Residential proxies are STRONGLY recommended. SimilarWeb blocks datacenter IPs aggressively.

Output Fields

Each result in the dataset contains the following fields:

Overview Data

Field	Type	Description
domain	string	The analyzed domain name.
domainName	string	Domain/site name as displayed on SimilarWeb.
globalRank	integer	Global traffic rank (parsed number). Null if unavailable.
globalRankDisplay	string	Global rank as displayed (e.g., "#1,234").
countryRank	integer	Country-specific traffic rank (parsed number).
countryRankDisplay	string	Country rank as displayed.
countryRankCountry	string	Country associated with the country rank.
categoryRank	integer	Category-specific traffic rank (parsed number).
categoryRankDisplay	string	Category rank as displayed.
categoryName	string	Website category (e.g., "Computers Electronics and Technology > Search Engines").
monthlyVisits	integer	Estimated total monthly visits (parsed number).
monthlyVisitsDisplay	string	Monthly visits as displayed (e.g., "1.2B", "345.6M").
bounceRate	string	Bounce rate percentage (e.g., "27.53%").
pagesPerVisit	string	Average pages viewed per visit (e.g., "8.72").
avgVisitDuration	string	Average visit duration (e.g., "00:10:23").
trafficSources	object	Traffic source percentages with keys: direct, referral, organicSearch, paidSearch, social, mail, displayAds.
topCountries	array	Top countries by traffic share. Each entry has country and percentage.
topReferringSites	array	Top referring websites. Each entry has site and percentage.
topDestinationSites	array	Top outgoing destination sites. Each entry has site and percentage.
competitors	array	Similar/competing websites. Each entry has site and affinity.
siteDescription	string	Website description from SimilarWeb.
url	string	The SimilarWeb analysis URL.
pageType	string	"overview" for main data, "competitors" for competitor-specific data.
scrapedAt	string	ISO 8601 timestamp of when the data was scraped.

Competitors Data (separate record)

Field	Type	Description
domain	string	The analyzed domain name.
competitors	array	Up to 20 competitors, each with site, affinity, category, and rank.
pageType	string	Always "competitors".
scrapedAt	string	ISO 8601 timestamp.

Example Output

{
    "domain": "github.com",
    "domainName": "github.com",
    "globalRank": 74,
    "globalRankDisplay": "#74",
    "countryRank": 42,
    "countryRankDisplay": "#42",
    "countryRankCountry": "United States",
    "categoryRank": 3,
    "categoryRankDisplay": "#3",
    "categoryName": "Computers Electronics and Technology > Programming and Developer Software",
    "monthlyVisits": 854200000,
    "monthlyVisitsDisplay": "854.2M",
    "bounceRate": "37.82%",
    "pagesPerVisit": "8.43",
    "avgVisitDuration": "00:07:12",
    "trafficSources": {
        "direct": 48.21,
        "referral": 7.45,
        "organicSearch": 32.18,
        "paidSearch": 0.12,
        "social": 5.87,
        "mail": 3.42,
        "displayAds": 0.01
    },
    "topCountries": [
        { "country": "United States", "percentage": 22.31 },
        { "country": "India", "percentage": 10.85 },
        { "country": "China", "percentage": 7.62 }
    ],
    "topReferringSites": [
        { "site": "stackoverflow.com", "percentage": 12.34 },
        { "site": "google.com", "percentage": 8.56 }
    ],
    "topDestinationSites": [
        { "site": "githubusercontent.com", "percentage": 15.67 }
    ],
    "competitors": [
        { "site": "gitlab.com", "affinity": "0.89" },
        { "site": "bitbucket.org", "affinity": "0.76" }
    ],
    "siteDescription": "GitHub is where over 100 million developers shape the future of software.",
    "url": "https://www.similarweb.com/website/github.com/",
    "pageType": "overview",
    "scrapedAt": "2024-01-15T10:30:00.000Z"
}

Example Use Cases

1. Competitive Intelligence: Compare your website's traffic metrics against competitors. Analyze their traffic sources, geographic reach, and engagement metrics to identify strategic advantages and gaps.

2. Market Research: Analyze traffic patterns for websites in a specific industry. Identify market leaders by global rank, understand regional preferences through geographic data, and discover traffic acquisition strategies.

3. Lead Generation for Agencies: Build prospect lists by analyzing potential client websites. Identify sites with low engagement metrics (high bounce rate, low pages per visit) that could benefit from your optimization services.

4. Investment Due Diligence: Evaluate web traffic for startups or e-commerce businesses. Cross-reference claimed traffic numbers with SimilarWeb estimates. Analyze traffic source diversity and growth trends.

5. SEO and Content Strategy: Understand where competitors get their traffic from. If a competitor gets 60% from organic search, study their content strategy. If another gets 30% from social, analyze their social media approach.

6. Partnership Evaluation: Before entering a partnership or advertising deal, verify a website's actual traffic volume and quality. Check geographic distribution to ensure audience alignment.

Cost Estimate

This actor is priced at the Premium tier: $0.003 per result.

Domains	Estimated Cost
10	$0.03
50	$0.15
100	$0.30
500	$1.50
1,000	$3.00

Notes

• SimilarWeb is heavily JavaScript-rendered and protected by sophisticated anti-bot measures. Residential proxies are required for reliable scraping. Datacenter proxies will almost certainly be blocked.
• The actor uses puppeteer-extra with stealth plugin and randomized viewports, user agents, and delays to minimize detection.
• SimilarWeb frequently changes its DOM structure and CSS class names. The actor uses multiple fallback selectors for each data point to maximize extraction reliability.
• Free tier SimilarWeb pages may show limited data compared to paid accounts. Some metrics may not be available for low-traffic websites.
• Traffic numbers are SimilarWeb's estimates and may differ from actual analytics data. They are best used for relative comparisons rather than absolute numbers.
• The actor runs with low concurrency (max 2 pages at a time) to reduce the chance of rate limiting.
• Some fields may be null or empty if SimilarWeb does not have sufficient data for a given domain.