Similarweb Scraper — Website Traffic, AI Traffic & Competitor Analysis

MCP and AI agent guide

Retrieve estimated website traffic and competitor intelligence. Use overview for a first domain check; choose other modes only when their extra report is needed.

Connect an authenticated Apify MCP client to:

https://mcp.apify.com/?tools=apivault_labs~similarweb-scraper

Use your own Apify token in the client Authorization header; never paste it into Actor inputs, prompts or public examples. Call tools/list and use the current schema for apivault_labs--similarweb-scraper. The input description summarizes the first-run/output contract; this README contains the full workflow.

Minimal Actor input:

{
  "mode": "overview",
  "domains": [
    "apify.com"
  ],
  "includeRawData": false,
  "includeScreenshot": false,
  "maxConcurrency": 1
}

For a direct Actor MCP call, add waitSecs: 45 to the tool arguments if the discovered schema supports it. waitSecs is a tool wait, not an Actor input, run timeout or spending limit. If the response is still RUNNING, keep its runId and poll get-actor-run; do not launch a duplicate paid run. After a network timeout, reconcile the existing run before repeating the call.

Use returned default storage IDs. Read get-dataset-items with a small limit (start with 5) and selected fields; use get-key-value-store-record only for the records supported below. Fetch a small page first, then paginate the Dataset instead of filling model context with the whole export. The wrapper may return a preview before the run finishes. Confirm terminal status, actual rows and result validity before reporting success. A missing record is not a zero value.

Agent decisions and output contract

• Start with mode=overview and one domain. compare uses the first domain as the primary website; similar_sites returns competitor lists; all_in_one requests extra enrichment.
• monitor needs a consistent, user-specific monitorKey; read MONITOR_SUMMARY. A first baseline is not evidence of a historical change. onlyChanges=true may legitimately return zero rows. Do not share the same key across unrelated clients.
• market_map writes MARKET_MAP. discovery uses discoveryType, discoveryCountry, optional discoveryCategory, and maxResults; it can run without input domains. Extra enrichment can take longer and may be unavailable.
• Each row has domain, processing_status, has_data and coverage/confidence fields. Treat no_data as missing coverage and failed as a failed lookup; neither means zero real traffic. Optional metrics and enrichment may be absent or null.
• Read OUTPUT and FAILED_DOMAINS from the default Key-Value Store. COMPARISON, MONITOR_SUMMARY, MARKET_MAP and DISCOVERY_SUMMARY exist only for their workflows. Do not request an invented SUMMARY or ERRORS record.
• Traffic is an estimate for the period provided in the result, not a live visitor counter or first-party analytics. Keep period/date information when comparing domains.
• Useful traffic results use the result pricing event; diagnostic no_data/failed rows can be present without a result event. Dataset row count therefore need not equal paid results. Platform usage is separately charged according to the active Pricing tab.

Use the smallest request that answers the user's question. Costs and source availability are separate from MCP discovery: seeing a schema does not prove a successful collection or image generation. Full instructions may be shortened by discovery summaries; open this Store README when more detail is needed.

Fast Similarweb scraper and website traffic API alternative for competitive intelligence, SEO research and market analysis. Enter one website or up to 10,000 domains to get monthly visits, rankings, engagement, traffic sources, top countries, keywords, competitors and AI/LLM referral insights.

No external analytics account or API credentials are required. Export clean, structured results to JSON, CSV or Excel, or connect the Actor to n8n, Make, Zapier, dashboards and data warehouses.

Why choose this Similarweb scraper?

• Mega-fast bulk website traffic analysis — lightweight parallel processing for up to 10,000 domains.
• Deeper AI traffic intelligence — AI referral share, estimated visits, source ranking, three-month source trends and available audience prompts.
• All-in-one competitor research — traffic analytics, similar websites, domain registration insights, technology signals, social profiles and keyword insights.
• Seven workflows in one Actor — analyze, compare, monitor, map markets and discover top or trending websites.
• Predictable bulk output — one clearly marked result per valid domain, stable ordering and retry-ready failures.
• Automation-ready website traffic API — structured fields and focused dataset views for production workflows.

Seven modes

1. Traffic Overview

The fastest mode for lists of up to 10,000 domains:

• monthly visits and three-month history;
• month-over-month and period growth;
• global, country and category ranks;
• bounce rate, pages per visit and visit duration;
• complete acquisition-channel split;
• organic, paid and AI visit estimates;
• top countries and organic keywords;
• AI/LLM referral share, source ranking and chatbot distribution;
• three-month AI-source trends and available top audience prompts.

2. Similar Sites

Find up to 20 competitors and alternatives for each domain, including:

• similarity rank and score;
• estimated monthly visits;
• category and country rank;
• description and preview URL.

3. Complete Profile

Combines traffic and competitors with expanded domain and company intelligence:

• registration date, domain age, expiry, registrar and nameservers;
• DNSSEC and domain status;
• company name, logo and favicon;
• public social profiles;
• detected CMS, e-commerce platform, frameworks, analytics and marketing tools;
• website keyword insights and visible content metrics.

4. Compare Websites

The first input domain becomes the primary website. Every output row receives:

• traffic, growth and AI rank within the batch;
• share of combined traffic;
• traffic gap and relative size versus the primary website;
• acquisition-channel gaps;
• leader, challenger, mid-pack or niche positioning.

A compact cohort report is saved separately as COMPARISON without creating an additional dataset row.

5. Monitor Changes

Use the same monitorKey in recurring or scheduled runs. The Actor remembers the previous baseline and reports:

• traffic and rank changes;
• acquisition-channel shifts;
• added or removed keywords;
• added or removed competitors;
• technology changes in Complete Profile monitoring.

Choose low, medium or high sensitivity. Enable onlyChanges to keep unchanged domains out of the dataset.

6. Market Map

Enter one or more seed websites. The Actor discovers their strongest competitors, enriches the selected domains and returns:

• market-map nodes and seed relationships;
• connection count per website;
• mapped traffic share;
• traffic, growth, rank, category and AI metrics;
• graph edges in the MARKET_MAP output record.

7. Website Discovery

No domain list is required. Discover:

• top websites worldwide or within a country;
• category leaders;
• rising and falling websites;
• new top-list entries and recently dropped websites.

Enable enrichment to add traffic, growth, channels, keywords and AI metrics to every discovered domain.

Quick start

Fast traffic check

{
  "mode": "overview",
  "domains": ["apify.com", "shopify.com", "openai.com"]
}

Complete competitor profiles

{
  "mode": "all_in_one",
  "domains": ["apify.com", "scraperapi.com"],
  "maxConcurrency": 3
}

Side-by-side comparison

{
  "mode": "compare",
  "domains": ["apify.com", "scraperapi.com", "brightdata.com"]
}

Weekly monitoring

{
  "mode": "monitor",
  "domains": ["apify.com", "scraperapi.com"],
  "monitorKey": "scraping-platforms",
  "changeThreshold": "medium",
  "onlyChanges": true
}

Competitive market map

{
  "mode": "market_map",
  "domains": ["apify.com"],
  "marketMapMaxCompetitors": 20
}

Top e-commerce websites in the United States

{
  "mode": "discovery",
  "discoveryType": "top",
  "discoveryCountry": "united-states",
  "discoveryCategory": "e-commerce-and-shopping",
  "enrichDiscovery": true
}

Trending websites

{
  "mode": "discovery",
  "discoveryType": "trending",
  "discoveryCountry": "united-states",
  "enrichDiscovery": true
}

Outputs

• Dataset — one structured row per returned website. By default, every unique valid input domain receives either data or a clearly marked status row.
• OUTPUT — run statistics and mode-specific summary.
• FAILED_DOMAINS — invalid, unavailable or zero-data domains ready for retry.
• COMPARISON — cohort rankings and leaders.
• MONITOR_SUMMARY — baseline and changed-domain counts.
• MARKET_MAP — graph edges and mapped-market summary.
• DISCOVERY_SUMMARY — discovery settings and result count.

Focused dataset views make Overview, Acquisition & AI, Competitors, Complete Profile, Comparison, Monitoring, Market Map and Discovery easy to inspect in the Apify Console.

Reliability and scale

• Automatic URL normalization and deduplication.
• Stable output order for bulk input reconciliation.
• Explicit success, no_data or failed processing status.
• Field-level coverage flags instead of ambiguous empty values.
• Failed-domain output for partial reruns.
• Lightweight Overview mode for high-volume enrichment.
• Explicit limits for market-map expansion.
• Run summaries include duplicate count, status totals and data-completeness percentage.

Set returnFailureRecords to false if you prefer successful dataset rows only. Retry-ready failures remain available separately in FAILED_DOMAINS.

Availability varies by website. Traffic figures are estimates rather than first-party analytics. Smaller or recently launched domains may have limited data.

FAQ

Do I need to connect an external analytics account?
No.

How many domains can Overview process?
Up to 10,000 unique domains per run.

Are competitors available for every website?
Coverage is broad, but some new or very small domains may have no related-site data.

Does monitoring work with Apify schedules?
Yes. Schedule the same input and keep the same monitorKey.

Is this an official Similarweb product?
No. This is an independent Actor and is not affiliated with or endorsed by Similarweb. Similarweb is a trademark of its respective owner.