Similarweb Scraper — Website Traffic, AI Traffic & Competitor Analysis

MCP and AI agent guide

Retrieve estimated website traffic and competitor intelligence. Use overview for a first domain check; choose other modes only when their extra report is needed.

Connect an authenticated Apify MCP client to:

https://mcp.apify.com/?tools=apivault_labs~similarweb-scraper

Use your own Apify token in the client Authorization header; never paste it into Actor inputs, prompts or public examples. Call tools/list and use the current schema for apivault_labs--similarweb-scraper. The input description summarizes the first-run/output contract; this README contains the full workflow.

Minimal Actor input:

{
  "mode": "overview",
  "domains": [
    "apify.com"
  ],
  "includeRawData": false,
  "includeScreenshot": false,
  "maxConcurrency": 1
}

For a direct Actor MCP call, add waitSecs: 45 to the tool arguments if the discovered schema supports it. waitSecs is a tool wait, not an Actor input, run timeout or spending limit. If the response is still RUNNING, keep its runId and poll get-actor-run; do not launch a duplicate paid run. After a network timeout, reconcile the existing run before repeating the call.

Use returned default storage IDs. Read get-dataset-items with a small limit (start with 5) and selected fields; use get-key-value-store-record only for the records supported below. Fetch a small page first, then paginate the Dataset instead of filling model context with the whole export. The wrapper may return a preview before the run finishes. Confirm terminal status, actual rows and result validity before reporting success. A missing record is not a zero value.

Agent decisions and output contract

• Start with mode=overview and one domain. compare uses the first domain as the primary website; similar_sites returns competitor lists; all_in_one requests extra enrichment.
• monitor needs a consistent, user-specific monitorKey; read MONITOR_SUMMARY. A first baseline is not evidence of a historical change. onlyChanges=true may legitimately return zero rows. Do not share the same key across unrelated clients.
• market_map writes MARKET_MAP. discovery uses discoveryType, discoveryCountry, optional discoveryCategory, and maxResults; it can run without input domains. Extra enrichment can take longer and may be unavailable.
• Each row has domain, processing_status, has_data and coverage/confidence fields. Treat no_data as missing coverage and failed as a failed lookup; neither means zero real traffic. Optional metrics and enrichment may be absent or null.
• Read OUTPUT and FAILED_DOMAINS from the default Key-Value Store. COMPARISON, MONITOR_SUMMARY, MARKET_MAP and DISCOVERY_SUMMARY exist only for their workflows. Do not request an invented SUMMARY or ERRORS record.
• Traffic is an estimate for the period provided in the result, not a live visitor counter or first-party analytics. Keep period/date information when comparing domains.
• Useful traffic results use the result pricing event; diagnostic no_data/failed rows can be present without a result event. Dataset row count therefore need not equal paid results. Platform usage is separately charged according to the active Pricing tab.

Use the smallest request that answers the user's question. Costs and source availability are separate from MCP discovery: seeing a schema does not prove a successful collection or image generation. Full instructions may be shortened by discovery summaries; open this Store README when more detail is needed.

Fast Similarweb scraper and website traffic API alternative for competitive intelligence, SEO research and market analysis. Enter one website or up to 10,000 domains to get monthly visits, rankings, engagement, traffic sources, top countries, keywords, competitors and AI/LLM referral insights.

No external analytics account or API credentials are required. Export clean, structured results to JSON, CSV or Excel, or connect the Actor to n8n, Make, Zapier, dashboards and data warehouses.

Why choose this Similarweb scraper?

• Mega-fast bulk website traffic analysis — lightweight parallel processing for up to 10,000 domains.
• Deeper AI traffic intelligence — AI referral share, estimated visits, source ranking, three-month source trends and available audience prompts.
• All-in-one competitor research — traffic analytics, similar websites, domain registration insights, technology signals, social profiles and keyword insights.
• Seven workflows in one Actor — analyze, compare, monitor, map markets and discover top or trending websites.
• Predictable bulk output — one clearly marked result per valid domain, stable ordering and retry-ready failures.
• Automation-ready website traffic API — structured fields and focused dataset views for production workflows.

Seven modes

1. Traffic Overview

The fastest mode for lists of up to 10,000 domains:

• monthly visits and three-month history;
• month-over-month and period growth;
• global, country and category ranks;
• bounce rate, pages per visit and visit duration;
• complete acquisition-channel split;
• organic, paid and AI visit estimates;
• top countries and organic keywords;
• AI/LLM referral share, source ranking and chatbot distribution;
• three-month AI-source trends and available top audience prompts.

2. Similar Sites

Find up to 20 competitors and alternatives for each domain, including:

• similarity rank and score;
• estimated monthly visits;
• category and country rank;
• description and preview URL.

3. Complete Profile

Combines traffic and competitors with expanded domain and company intelligence:

• registration date, domain age, expiry, registrar and nameservers;
• DNSSEC and domain status;
• company name, logo and favicon;
• public social profiles;
• detected CMS, e-commerce platform, frameworks, analytics and marketing tools;
• website keyword insights and visible content metrics.

4. Compare Websites

The first input domain becomes the primary website. Every output row receives:

• traffic, growth and AI rank within the batch;
• share of combined traffic;
• traffic gap and relative size versus the primary website;
• acquisition-channel gaps;
• leader, challenger, mid-pack or niche positioning.

A compact cohort report is saved separately as COMPARISON without creating an additional dataset row.

5. Monitor Changes

Use the same monitorKey in recurring or scheduled runs. The Actor remembers the previous baseline and reports:

• traffic and rank changes;
• acquisition-channel shifts;
• added or removed keywords;
• added or removed competitors;
• technology changes in Complete Profile monitoring.

Choose low, medium or high sensitivity. Enable onlyChanges to keep unchanged domains out of the dataset.

6. Market Map

Enter one or more seed websites. The Actor discovers their strongest competitors, enriches the selected domains and returns:

• market-map nodes and seed relationships;
• connection count per website;
• mapped traffic share;
• traffic, growth, rank, category and AI metrics;
• graph edges in the MARKET_MAP output record.

7. Website Discovery

No domain list is required. Discover:

• top websites worldwide or within a country;
• category leaders;
• rising and falling websites;
• new top-list entries and recently dropped websites.

Enable enrichment to add traffic, growth, channels, keywords and AI metrics to every discovered domain.

Quick start

Fast traffic check

{
  "mode": "overview",
  "domains": ["apify.com", "shopify.com", "openai.com"]
}

Complete competitor profiles

{
  "mode": "all_in_one",
  "domains": ["apify.com", "scraperapi.com"],
  "maxConcurrency": 3
}

Side-by-side comparison

{
  "mode": "compare",
  "domains": ["apify.com", "scraperapi.com", "brightdata.com"]
}

Weekly monitoring

{
  "mode": "monitor",
  "domains": ["apify.com", "scraperapi.com"],
  "monitorKey": "scraping-platforms",
  "changeThreshold": "medium",
  "onlyChanges": true
}

Competitive market map

{
  "mode": "market_map",
  "domains": ["apify.com"],
  "marketMapMaxCompetitors": 20
}

Top e-commerce websites in the United States

{
  "mode": "discovery",
  "discoveryType": "top",
  "discoveryCountry": "united-states",
  "discoveryCategory": "e-commerce-and-shopping",
  "enrichDiscovery": true
}

Trending websites

{
  "mode": "discovery",
  "discoveryType": "trending",
  "discoveryCountry": "united-states",
  "enrichDiscovery": true
}

Outputs

• Dataset — one structured row per returned website. By default, every unique valid input domain receives either data or a clearly marked status row.
• OUTPUT — run statistics and mode-specific summary.
• FAILED_DOMAINS — invalid, unavailable or zero-data domains ready for retry.
• COMPARISON — cohort rankings and leaders.
• MONITOR_SUMMARY — baseline and changed-domain counts.
• MARKET_MAP — graph edges and mapped-market summary.
• DISCOVERY_SUMMARY — discovery settings and result count.

Focused dataset views make Overview, Acquisition & AI, Competitors, Complete Profile, Comparison, Monitoring, Market Map and Discovery easy to inspect in the Apify Console.

Reliability and scale

• Automatic URL normalization and deduplication.
• Stable output order for bulk input reconciliation.
• Explicit success, no_data or failed processing status.
• Field-level coverage flags instead of ambiguous empty values.
• Failed-domain output for partial reruns.
• Lightweight Overview mode for high-volume enrichment.
• Explicit limits for market-map expansion.
• Run summaries include duplicate count, status totals and data-completeness percentage.

Set returnFailureRecords to false if you prefer successful dataset rows only. Retry-ready failures remain available separately in FAILED_DOMAINS.

Availability varies by website. Traffic figures are estimates rather than first-party analytics. Smaller or recently launched domains may have limited data.

FAQ

Do I need to connect an external analytics account?
No.

How many domains can Overview process?
Up to 10,000 unique domains per run.

Are competitors available for every website?
Coverage is broad, but some new or very small domains may have no related-site data.

Does monitoring work with Apify schedules?
Yes. Schedule the same input and keep the same monitorKey.

Is this an official Similarweb product?
No. This is an independent Actor and is not affiliated with or endorsed by Similarweb. Similarweb is a trademark of its respective owner.

Match website traffic research to the right workflow

Use this website traffic scraper for estimated audience and competitor research, with a clear period and coverage check for every domain. Overview is the small starting point; expanded profiles and discovery answer different questions and can require more requests. A Similarweb traffic estimate is not your site's first-party analytics or a live count of visitors.

Question	Workflow	Quality check
What traffic context is available?	overview	has_data, period and metric coverage
How do known competitors compare?	compare	Same reporting context and known missing domains
Which related sites should I review?	similar_sites	Similarity is not a verified commercial relationship
Did a saved cohort change?	monitor	Stable monitorKey and an established baseline

Check one domain with minimal enrichment

{
  "mode": "overview",
  "domains": [
    "apify.com"
  ],
  "includeSimilarSites": false,
  "includeTopKeywords": false,
  "includeAiTraffic": false,
  "returnFailureRecords": false,
  "includeNoData": false,
  "maxConcurrency": 1,
  "maxRetries": 0
}

Read OUTPUT and FAILED_DOMAINS even when the Dataset is empty. Omitted diagnostic rows do not turn a failed lookup into a successful one.

Compare a small known cohort

{
  "mode": "compare",
  "domains": [
    "apify.com",
    "scraperapi.com"
  ],
  "maxConcurrency": 2,
  "maxRetries": 0
}

The first domain is the primary website. Traffic share is within this submitted cohort, not the entire market. Retain no_data and failed coverage notes before interpreting a relative rank.

Create a named baseline for scheduled monitoring

{
  "mode": "monitor",
  "domains": [
    "apify.com"
  ],
  "monitorKey": "my-team-public-site-watchlist",
  "onlyChanges": false,
  "maxConcurrency": 1
}

Choose your own consistent monitorKey for this watchlist. The first run stores the baseline; later runs provide observed differences. A zero-row onlyChanges run can be expected rather than a collection error.

Review a bounded discovery sample

{
  "mode": "discovery",
  "discoveryType": "top",
  "discoveryCountry": "united-states",
  "maxResults": 3,
  "enrichDiscovery": false
}

Discovery lists and groups have source coverage limits. Disabling enrichment changes which metrics are available. This sample does not prove every website in a country or a complete historical ranking.

Traffic API costs and defensible comparisons

Current result-event rates are $2.50 per 1,000 on Free and $1 per 1,000 on paid Apify plans. Platform resources are additional; there is no configured Actor-start event. Two useful paid-plan records cost $0.002 in result events before resources. Diagnostic failed/no_data records may appear in the Dataset without a result event, so Dataset length alone is not a billing calculation. Use settled events and the current Pricing tab.

Use the smallest workflow that answers the question. Complete profiles, competitor expansion, market mapping and enriched discovery can increase work even if the returned cohort is small. A maxResults value is a discovery ceiling, not a global limit for every workflow. Increasing concurrency or retries cannot guarantee data for a small site with no published coverage.

For a dashboard, join on normalized domain and retain processing_status, has_data, the reported period and field coverage. Keep unknown metrics distinct from numeric zero. Acquisition shares and modeled AI-referral visits inherit the source coverage and estimation assumptions; they do not establish actual sales attribution, identifiable visitors or customer conversations. Available prompts should not be presented as a complete log of users' private AI chats.

JSON preserves nested histories and channel breakdowns; a CSV projection is convenient for selected scalar metrics. In an AI or n8n workflow, poll the same run and read OUTPUT/FAILED_DOMAINS, plus the mode-specific summary. There is no generic SUMMARY/ERRORS contract for this Actor.

Can I compare a missing-data site as zero traffic? No. no_data means unavailable coverage. Exclude or explicitly mark it before ranking a cohort.

Does monitoring prove continuous growth? It compares available saved observations. Period changes and source revisions must be considered alongside the measured difference between those observations.

n8n and developer integrations

Use this Actor in your automation or application with the existing community integrations below. An Apify account and API token are required; Actor and platform charges follow the current pricing.

• n8n community node: n8n-nodes-apivault-similarweb. Install package name n8n-nodes-apivault-similarweb in a self-hosted n8n instance that permits community packages.
• Python client and examples: GitHub.
• n8n source and usage: GitHub.

These are community packages. Publication on npm does not establish n8n verification or availability in n8n Cloud. Review the linked package README for supported inputs and setup.