Similarweb Scraper — website traffic checker, competitor analysis & AI visibility

A website traffic checker and website analytics tool for whole lists of domains. Collect monthly visits, growth, ranks, acquisition channels, keywords, AI referrals, WHOIS, company contacts and homepage technologies. Run competitor analysis side by side or build a competitive market map.

🤖 AI visibility per assistant: every tracked site gets its own ChatGPT, Gemini, Perplexity, Claude, Copilot, DeepSeek and Grok share columns, so you can sort a list by who sends AI traffic.

No Similarweb login or browser session is required. Existing API integrations using base_data, similar_sites, aitdk, or all remain compatible.

▶️ Your first run

1. Open Input. The first question is What do you want to know? — leave it on 📊 Traffic & audience, which covers traffic sources, engagement, ranks, top countries, search keywords and AI referrals.
2. Under Websites, add the sites you want to analyze, one per entry. A domain such as apify.com or a full HTTP(S) website URL is accepted. Duplicate websites are removed automatically.
3. Optional workflow settings apply to Map a market or Everything we can get. Advanced has sensible defaults. Click Start. For a first run, use 1–3 websites.
4. Open Output → Results → Overview for the headline metrics — traffic, growth, rank, category, top country, top channel and AI-referral share. Switch to Traffic, Channels, Countries or Keywords for details.
5. Use Export to download the data. Choose All fields when you want the full record. JSON preserves nested objects; list cells in the table expand to show their entries.

Copy this into Input → JSON for a first run:

{
  "domains": ["python.org", "docker.com"],
  "mode": "base_data"
}

Choose ⭐ Everything we can get to add competitor discovery, WHOIS and domain age, homepage tech stack and company contacts. Each website produces one main result. The countries, keywords and other lists inside that record are included in its collection event.

📋 Ready-made tasks

Prefer to start from a working setup? Each task below is a saved input you can run as is, then swap in your own websites.

Task	Workflow	Websites
Compare ecommerce website traffic	⚔️ Compare	amazon.com, ebay.com, walmart.com, etsy.com, temu.com
Compare travel booking traffic	⚔️ Compare	booking.com, airbnb.com, expedia.com, tripadvisor.com, agoda.com
Compare job board traffic	⚔️ Compare	indeed.com, glassdoor.com, linkedin.com, ziprecruiter.com, monster.com
Research SaaS website traffic	📊 Traffic & audience	notion.so, asana.com, monday.com, clickup.com, trello.com
Research developer website audiences	📊 Traffic & audience	github.com, stackoverflow.com, gitlab.com, dev.to
Analyze AI referrals to tech sites	📊 Traffic & audience	openai.com, anthropic.com, huggingface.co, perplexity.ai
Discover Shopify competitors	🧭 Find competitors	shopify.com, woocommerce.com
Map design software competitors	🕸️ Map a market	canva.com, figma.com
Research ecommerce domain data	⭐ Everything we can get	shopify.com, bigcommerce.com, wix.com

🚀 Pick your goal

Workflow (as it appears in the input)	Best for	What you receive	Collection event
📊 Traffic & audience (base_data)	Fast website overview	Traffic history, growth, ranks, engagement, channels, geography, keywords, AI visibility, screenshot	base-data
🧭 Find competitors (similar_sites)	Competitor discovery	Similar websites, similarity grades, estimated visits, categories, tags, related apps	similar-sites
🔍 SEO keywords & tech stack (aitdk)	Domain due diligence	WHOIS/RDAP, domain age, expiry, homepage keyword density, company metadata, social links, detected technologies	whois-keywords
⭐ Everything we can get (all)	One row with everything	Traffic, competitor discovery, WHOIS and domain age, homepage tech stack and company contacts, merged into a single row	all-in-one
⚔️ Compare websites (compare)	Side-by-side benchmark	Actual traffic shares, leaders, ranks, engagement position, and gaps versus the first domain	base-data
🕸️ Map a market (market_map)	Competitive landscape	Seed sites, auto-discovered competitors, enriched traffic rows, market shares, and a separate relationship dataset	base-data

Current rates are listed in the Actor's Pricing tab. Successful and partial records use the workflow's collection event. Failed lookups go to Errors without a result-event charge; the platform's Actor start event still applies.

🏢 Company & contact details — included in “Everything we can get”

⭐ Everything we can get (all) and 🔍 SEO keywords & tech stack (aitdk) read the website's own public homepage and return the business details found there, inside the same result event, at no extra charge:

Field	What it holds
companyName	Organization name from structured homepage data
companyFoundingDate	Founding year, when the homepage declares it
companyAddress	Postal address — street, locality, region, postal code, country
contactEmails	Contact emails from mailto: links and structured data, deduplicated
contactPhones	Phone numbers from tel: links and structured data, deduplicated
socialLinks	Official LinkedIn, X/Twitter, Facebook, Instagram, YouTube, TikTok and GitHub profiles
technologies, technologyCount	Technology signatures detected on the homepage
whois, domainAgeDays, daysUntilDomainExpiration	Registrar, dates, nameservers and derived domain age

These come only from what the homepage publishes, so a bare landing page returns fewer of them. Open Results → Contacts for the contact and firmographic view, and Results → Website for the full domain record.

{
  "domains": ["shopify.com", "stripe.com"],
  "mode": "all"
}

Extra audience layer (optional)

includeInDepth: true additionally reads Similarweb's own public website page for audience age and gender, referring domains, ad publishers and rank history. It is off by default because that page sits behind extra protection and adds time to the run. When it cannot be read, the Actor stops asking after the first attempts instead of slowing the run, and every website still comes back with all of the traffic, competitor, domain and contact data above. indepthStatus reports whether the layer was collected.

{
  "domains": ["python.org", "docker.com"],
  "mode": "all",
  "includeInDepth": true
}

🎯 Popular use cases

You want to…	Use	Look at
Size up a prospect or partner before a call	📊 Traffic & audience	latestMonthlyVisits, trafficTrend, topCountryCode, topTrafficChannel
Qualify a lead list by website size	📊 Traffic & audience + filters	trafficDataAvailable, minMonthlyVisits, maxGlobalRank
Build a competitor list for a niche	🧭 Find competitors or 🕸️ Map a market	similarSites, marketTrafficSharePercent, emergingCompetitor
Benchmark your site against rivals	⚔️ Compare websites	trafficSharePercent, trafficGapVsPrimaryPercent, winnerMetrics
See how much traffic AI assistants send	📊 Traffic & audience	aiTrafficPercent, aiChatgptSharePercent, topAiPlatform
Plan SEO and paid search	📊 Traffic & audience	organicSearchPercent, paidSearchPercent, topKeywords
Check a domain before buying or partnering	🔍 SEO keywords & tech stack	domainAgeDays, daysUntilDomainExpiration, whois, technologies
Enrich a CRM with company signals	⭐ Everything we can get	companyName, socialLinks in websiteIntelligence, category

✨ What makes the output comfortable

• Flat columns where they matter: latestMonthlyVisits, trafficChangePercent, bounceRatePercent, organicSearchPercent, and dozens more export cleanly to CSV and Excel.
• Original objects remain: existing consumers can still use engagement, estimatedMonthlyVisits, trafficSources, countryRank, and other legacy fields.
• Output destinations: Results, Final comparison, Final market analysis, Market links, Errors and Run summary, plus focused topic exports.
• Results appear as they finish: independent website profiles are saved immediately. Compare and Market Map calculate their cohort-wide shares after collecting the bounded comparison group.
• Clean failures: no error rows mixed into paid result counts or downstream exports.
• Clear finish state: every run writes an OUTPUT JSON summary with exact result, error, filter, duration, and charged-event counts.
• Honest enrichment: unavailable values stay empty. The Actor does not invent demographics, traffic, technologies, or company data.

📏 How much data each website returns

List	Depth	Workflow
Monthly visits (trafficHistory, estimatedMonthlyVisits)	Last 3 months the source publishes	Traffic & audience, Everything
Top countries (topCountryShares)	Top 5	Traffic & audience, Everything
Search keywords (topKeywords)	Top 5, with volume and CPC	Traffic & audience, Everything
Traffic channels	All 10 channels	Traffic & audience, Everything
AI assistants (aiChatbotDistribution)	Every assistant the source reports	Traffic & audience, Everything
Similar websites (similarSites)	Up to 20	Find competitors, Everything, Map a market
Homepage keyword density	Top 20 words	SEO keywords & tech stack, Everything
Company contacts	Emails, phones, address, social profiles	SEO keywords & tech stack, Everything

Similarweb publishes monthly figures. The latest month usually appears in the first half of the following month; snapshotDate tells you which month a row describes.

📊 Traffic & audience fields

The base_data workflow returns the source-compatible objects plus useful derived fields.

Identity and rank

domain, websiteUrl, similarwebUrl, siteName, title, description, category, snapshotDate, screenshot, globalRank, countryRank, countryCode, countryRankValue, categoryRank, categoryRankValue, marketPosition

Traffic and engagement

engagement, estimatedMonthlyVisits, trafficHistory, latestMonthlyVisits, previousMonthlyVisits, trafficChangeMoM, trafficChangePercent, threeMonthGrowthPercent, trafficTrend, bounceRatePercent, pagesPerVisit, timeOnSiteSeconds

Metric	Meaning and units
latestMonthlyVisits	Estimated visits in the latest available month, not unique visitors
engagement.month, engagement.year	Period of the engagement metrics
bounceRatePercent	Bounce rate on a 0–100 scale; engagement.bounceRate preserves the original 0–1 fraction
pagesPerVisit	Average pages per visit, not total pageviews
timeOnSiteSeconds	Average visit duration in seconds
trafficChangePercent	Calculated month-over-month change; trafficChangeMoM retains the fractional form

Use the dates in trafficHistory and snapshotDate when comparing periods. An unavailable metric stays empty; a real zero stays zero.

Acquisition channels

trafficSources, directPercent, organicSearchPercent, paidSearchPercent, organicSocialPercent, paidSocialPercent, referralsPercent, mailPercent, displayAdsPercent, affiliatePercent, aiTrafficPercent, topTrafficChannel, topTrafficChannelPercent, organicPaidRatio, trafficConcentrationScore, acquisitionDiversityScore

The Channels view separates organic and paid search, organic and paid social, direct visits, referrals, email, display advertising, affiliates, and AI referrals. Flat *Percent fields use 0–100; values inside trafficSources use 0–1. Referral share measures the channel's contribution, not a list of referring URLs.

Geography, search, AI, and competitors

topCountryShares, topCountryCode, topCountrySharePercent, topKeywords, topKeyword, topKeywordVolume, aiTraffic, aiChatbotDistribution, aiTotalVisits, estimatedAiVisits, topAiPlatform, topAiPrompts, competitors, competitorCount, topCompetitor

Countries shows each available top country with countryCode, countryName, countryId, share, and sharePercent inside topCountryShares. Names are taken from the source's country dictionary. These are the leading countries, so their shares need not add up to 100%.

countryRank.countryCode / countryCode identify the country used for the country rank. topCountryCode identifies the first country in the audience breakdown. They can legitimately differ.

Keywords shows the leading term and an expandable topKeywords list with name, volume, cpc, and estimatedValue, as supplied by the source. Search-channel shares are split into organic and paid; the keyword list itself is not classified as OrganicKeywords or PaidKeywords. A CPC value does not prove that the website pays for that keyword. keywordDensity in Domain SEO & WHOIS is a separate analysis of words on the homepage, not search traffic.

For competitor discovery, use similar_sites or all: their similarSites lists include descriptions, estimated visits, categories and similarity signals. Similarity is a discovery signal, not a guarantee that every listed site is a direct business competitor.

Fields ending in Percent, such as topCountrySharePercent, use a 0–100 scale; the raw share inside topCountryShares uses a 0–1 fraction (0.4628 means 46.28%).

Illustrative country entry:

{
  "countryCode": "US",
  "countryId": 840,
  "countryName": "United States",
  "share": 0.2543,
  "sharePercent": 25.43
}

Quality signals

dataCompletenessScore, riskFlags, isSmall, trafficDataAvailable, coverage, status

Every domain you submit comes back as a row. Similarweb only measures websites above a traffic threshold, so a small local business usually has no traffic figures at all. That is an answer about the website, not a failure, and it is returned as a normal row with trafficDataAvailable: false, coverage: "not_tracked_by_similarweb" and status: "not_tracked". These rows are never charged. Filter on trafficDataAvailable to split a list into "measured by Similarweb" and "too small to be measured" — for lead lists, that split is often the point of the run.

Competitor discovery works the same way: a website with no similar-sites profile comes back with coverage: "not_tracked_by_similarsites" instead of disappearing. Compare and Map a market leave untracked websites out of the ranking, the baseline and the market shares — there is nothing to rank, watch or divide — but still return the row, free.

The legacy isDataFromGoogleAds key is retained for API compatibility only. It mirrors the source's IsDataFromGa flag and must not be used as evidence of Google Ads spending; use paidSearchPercent for paid-search traffic share.

riskFlags are deterministic signals such as traffic_decline, single_channel_dependence, high_bounce_rate, domain_expiring_soon, and limited_public_data. They are not an opaque AI score.

🧭 Competitor discovery fields

The similar_sites workflow returns:

{
  "domains": ["docker.com", "notion.so"],
  "mode": "similar_sites"
}

domain, title, description, category, categoryRank, totalVisits, thumbnail, screenshot, favicon, tags, similarSites, relatedApps, status

Each similarSites item can contain:

{
  "site": "peer-site.test",
  "description": "Example competitor website",
  "category": "Business_and_Consumer_Services",
  "similarityRank": 1,
  "topCountryRank": 2400,
  "totalVisits": 1850000,
  "grade": 0.91,
  "thumbnail": "https://example.test/example-preview.png"
}

Result fragments are illustrative. The runnable input examples use real websites.

🔍 Domain, company & technology fields

The aitdk workflow combines domain registration data with analysis of the public homepage. Open Results → Website to inspect it.

{
  "domains": ["python.org", "docker.com"],
  "mode": "aitdk"
}

• whois: registrar, status, nameservers, DNSSEC, registration, expiration, and last-change dates
• domainAgeDays, daysUntilDomainExpiration
• keywordDensity: page title, visible token count, top tokens, counts, and frequencies
• websiteIntelligence: page title, meta description, language, canonical URL, page image, company metadata, public social profiles, generator, and technology signals
• companyName, companyLogo, companyFoundingDate, companyAddress
• contactEmails, contactPhones, socialLinks
• technologies, technologyCount

Contact details come only from what the homepage publishes — mailto: and tel: links and structured Organization data — so they are precise but not exhaustive: a website that hides its contacts returns empty lists rather than guessed values. Emails and phones are deduplicated, and a phone written two ways (+1 415 555 0100 and +14155550100) is collapsed into one. companyAddress holds whichever of street, locality, region, postal code and country the homepage declares.

Technology detection is based on visible homepage signatures and is labelled as detected or declared; it is not presented as a guaranteed full technology stack. A company logo is returned only when structured organization data explicitly identifies it as a logo — a generic Open Graph hero image is not mislabeled.

WHOIS fields are filled from the available registry data. When a registry record is unavailable, other collected details remain in the result and notes explains the missing part.

⚔️ Compare websites

Put the primary website first. Profiles stream into Results. Open Final comparison to see each site's share, position and gap against that primary website after the collected group is ready.

{
  "domains": ["nytimes.com", "theguardian.com", "cnn.com"],
  "mode": "compare",
  "maxConcurrency": 10
}

Comparison fields include:

primaryDomain, compareRank, trafficSharePercent, trafficGapVsPrimaryPercent, globalRankGapVsPrimary, growthRank, engagementRank, competitivePositionScore, winnerMetrics

Illustrative result fragment:

{
  "domain": "competitor-a.test",
  "mode": "compare",
  "primaryDomain": "primary-site.test",
  "compareRank": 2,
  "trafficSharePercent": 31.42,
  "trafficGapVsPrimaryPercent": -38.7,
  "growthRank": 1,
  "engagementRank": 3,
  "winnerMetrics": ["growth"]
}

🕸️ Build a market map

Market Map starts with one or more seed domains, discovers related sites, deduplicates them, enriches the strongest candidates, and calculates share inside the returned landscape.

{
  "domains": ["docker.com"],
  "mode": "market_map",
  "marketMapMaxCompetitors": 2,
  "maxConcurrency": 10
}

Final market analysis contains the full website fields together with:

marketRole, sourceSeedDomains, similarityGrade, similarityRank, marketTrafficSharePercent, marketRank, emergingCompetitor

Open Results for live website profiles and Final market analysis for the completed market calculations. The separate Market links destination contains one edge per returned seed-to-competitor connection:

{
  "seedDomain": "seed-site.test",
  "competitorDomain": "peer-site.test",
  "similarityRank": 1,
  "grade": 0.91,
  "competitorVisits": 1850000,
  "competitorGlobalRank": 43210,
  "trafficChangePercent": 8.4,
  "marketTrafficSharePercent": 17.6,
  "category": "Business_and_Consumer_Services"
}

📥 Input reference

Field	Type	Default	Notes
domains	array of strings	required	Websites in the input. Domains or full URLs; normalized, deduplicated and validated
mode	string	base_data	Workflow in the input — one of the six above
includeInDepth	boolean	false	Everything we can get · try extra audience data. Audience age/gender, referrers and rank history; off by default. Other workflows ignore it
maxDomains	integer	10000	Stop if the list is longer than — guards against a huge pasted list
marketMapMaxCompetitors	integer	10	Map a market · how many competitors to add — up to 100
minMonthlyVisits	integer	empty	Keep websites with at least — minimum monthly visits
maxMonthlyVisits	integer	empty	…and at most — maximum monthly visits
maxGlobalRank	integer	empty	Global rank no worse than — keep sites inside this Similarweb rank
minTrafficChangePercent	integer	empty	Month-over-month growth of at least, in percent; negative values allowed
maxBounceRatePercent	integer	empty	Bounce rate no higher than, in percent
includeCategories	array of strings	[]	Categories to keep — matched against the Similarweb category
excludeCategories	array of strings	[]	Categories to skip
topCountries	array of strings	[]	Top visitor country is one of — two-letter codes
minCountrySharePercent	integer	empty	…bringing at least this share of visits — with topCountries, keeps sites whose listed countries together bring this share
maxCountryRank	integer	empty	Country rank no worse than — rank in the site's main country
maxCategoryRank	integer	empty	Category rank no worse than — rank in its Similarweb category
maxTrafficChangePercent	integer	empty	…and at most — upper limit for month-over-month change; negative finds declining sites
minPagesPerVisit	number	empty	Pages per visit at least
minTimeOnSiteSeconds	integer	empty	Time on site at least, in seconds
topTrafficChannels	array of strings	[]	Main traffic channel is one of — direct, organic_search, paid_search, organic_social, paid_social, referrals, email, display_ads, affiliate, ai
minOrganicSearchPercent	integer	empty	Organic search at least, in percent of visits
minPaidSearchPercent	integer	empty	Paid search at least, in percent of visits
minAiTrafficPercent	integer	empty	AI assistants at least, in percent of visits
minDomainAgeYears	number	empty	Domain at least this old, in years since WHOIS registration
maxDomainAgeYears	number	empty	…and at most this old
requireContactEmail	boolean	false	Only websites with a public email on the homepage
requireContactPhone	boolean	false	Only websites with a public phone on the homepage
includeTechnologies	array of strings	[]	Uses one of these technologies — matched against the detected tech stack
excludeTechnologies	array of strings	[]	Skip websites using these technologies
includeTlds	array of strings	[]	Domain ends with — for example .de, .co.uk
excludeTlds	array of strings	[]	Skip domain endings
maxConcurrency	integer	10	Websites at a time — 1–50
proxyConfiguration	object	Actor default	Proxy. Keep the default unless you have your own

🎯 Keep only the websites you want

Filters run on the finished record, before billing, so a website you filter out is never saved and never charged: size (minMonthlyVisits, maxMonthlyVisits), rank (maxGlobalRank, maxCountryRank, maxCategoryRank), growth (minTrafficChangePercent, maxTrafficChangePercent), quality (maxBounceRatePercent, minPagesPerVisit, minTimeOnSiteSeconds), topic (includeCategories, excludeCategories), audience (topCountries, minCountrySharePercent) and traffic channel (topTrafficChannels, minOrganicSearchPercent, minPaidSearchPercent, minAiTrafficPercent).

They pay off most where the Actor finds websites for you — Map a market and Find competitors — but work in every workflow that carries traffic data: Traffic & audience, Everything we can get, Compare and Map a market. SEO keywords & tech stack collects no traffic, so traffic filters stay inactive there and the run says so in inputWarnings.

A second group reads the WHOIS record and the homepage, so it applies to SEO keywords & tech stack and Everything we can get: domain age (minDomainAgeYears, maxDomainAgeYears), contacts (requireContactEmail, requireContactPhone) and tech stack (includeTechnologies, excludeTechnologies). Domain endings (includeTlds, excludeTlds) work in every workflow. If a homepage could not be read, requireContactEmail, requireContactPhone and includeTechnologies drop that website rather than charge for a row that may lack what you asked for; excludeTechnologies keeps it.

A website whose metric Similarweb does not publish is kept rather than dropped, so filtering never looks like data loss. Every run reports what it removed: filteredOut is the total, filterBreakdown counts the drops per setting, and filterHint names the costliest one in plain words, for example "18 of 40 website(s) were removed by your filters, 15 of them by minMonthlyVisits. To keep more, lower minMonthlyVisits."

📦 Outputs and table views

Choose Results to browse live website profiles, then switch topics using the tabs above the table. Overview starts with the headline metrics; detailed source fields remain available in the topic views and dataset export. Completed cohort calculations have their own final tables.

Destination	Contains
Results	Website records and their topic views
Final comparison	Full website fields plus comparison ranks, shares, scores and gaps
Final market analysis	Full website fields plus calculated market ranks and shares
Market links	Seed-to-competitor relationships from Market map
Errors	Failed lookups, stored separately and never charged as results
Run summary	Counts, timing, filters and workflow details

Within Results, use these topic tabs. Start with Competitors for competitor discovery and Website for Domain SEO; traffic metrics belong to workflows that collect traffic data.

Output	Contains
Overview	Website, category, monthly visits, growth, global rank, leading country, top channel, AI-referral share, bounce rate and status
Traffic	Traffic history, growth and engagement
Channels	Source percentages, concentration, and diversity
Countries	Leading country plus country names, codes and audience shares
Keywords	Leading term plus keyword text, volume, CPC and source-estimated value
AI traffic	AI share, chatbot distribution and available prompts
Competitors	Similar websites, direct competitors, tags and related apps
Contacts	Company name, founding year, postal address, emails, phones and social profiles
Website	Preview, category, company, WHOIS, technologies and collection details
Compare	Leaders, shares and gaps between compared websites
Market	Enriched seed and competitor websites

Contacts and Website come from SEO keywords & tech stack and Everything we can get. The Compare and Market presets are also available inside the final analytics destination. Switching views does not collect data or create additional charges. Export the complete final dataset to keep source fields and cohort calculations together; display presets do not remove stored values.

📤 Traffic & audience example

Illustrative result fragment showing the field structure:

{
  "domain": "demo-site.test",
  "mode": "base_data",
  "snapshotDate": "2026-07-01T00:00:00+00:00",
  "globalRank": 18420,
  "estimatedMonthlyVisits": {
    "2026-05-01": 220000,
    "2026-06-01": 238000,
    "2026-07-01": 245000
  },
  "latestMonthlyVisits": 245000,
  "previousMonthlyVisits": 238000,
  "trafficChangePercent": 2.94,
  "trafficTrend": "stable",
  "bounceRatePercent": 41.2,
  "pagesPerVisit": 3.84,
  "timeOnSiteSeconds": 176.5,
  "directPercent": 31.0,
  "organicSearchPercent": 46.0,
  "aiTrafficPercent": 2.1,
  "topTrafficChannel": "Organic search",
  "topCountryCode": "US",
  "topKeyword": "demo analytics",
  "topAiPlatform": "chatgpt.com",
  "dataCompletenessScore": 92,
  "riskFlags": [],
  "status": "ok"
}

Traffic values are estimates tied to the source snapshot date, not real-time analytics or exact first-party measurements.

🔌 API examples

Set the APIFY_TOKEN environment variable to your Apify API token. These examples use Bash syntax.

Start a run:

curl -X POST "https://api.apify.com/v2/acts/trakk~similarweb-scraper/runs" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["python.org","docker.com"],"mode":"base_data","maxConcurrency":2}'

Run synchronously and receive dataset items:

curl -X POST "https://api.apify.com/v2/acts/trakk~similarweb-scraper/run-sync-get-dataset-items" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["nytimes.com","theguardian.com","cnn.com"],"mode":"compare"}'

Python, with the official apify-client package:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("trakk/similarweb-scraper").call(run_input={
    "domains": ["python.org", "docker.com"],
    "mode": "base_data",
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["domain"], item.get("latestMonthlyVisits"), item.get("topTrafficChannel"))

JavaScript / Node.js, with apify-client:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('trakk/similarweb-scraper').call({
    domains: ['nytimes.com', 'theguardian.com', 'cnn.com'],
    mode: 'compare',
});
const summaryRecord = await client.keyValueStore(run.defaultKeyValueStoreId).getRecord('OUTPUT');
const analysisId = summaryRecord.value.details.analyticsDatasetId;
const { items } = await client.dataset(analysisId).listItems();
console.table(items.map((i) => ({ domain: i.domain, share: i.trafficSharePercent, rank: i.compareRank })));

For larger jobs, start an asynchronous run and consume the output URLs returned with the run. Dataset items can be exported as JSON, JSONL, CSV, Excel, XML, HTML, or RSS.

📅 Schedules, webhooks & integrations

• Alerts: add a webhook on Run succeeded to post the dataset URL to Slack, email or your own endpoint.
• Spreadsheets: connect the Actor to Google Sheets, Make, Zapier or n8n through the Apify integrations and map flat columns such as latestMonthlyVisits or organicSearchPercent straight into a sheet.
• AI agents: the Actor can be called as a tool through the Apify MCP server, so an assistant can look up website traffic on demand.

⚙️ Following a run

• Completed website profiles appear in the dataset during collection.
• Compare and Market map calculate shares across the collected group before saving their final rows.
• Run summary shows saved results, partial records, errors, filtered checks and elapsed time.
• ok identifies a successful collection; partial keeps the available data and explains missing parts in notes. Failed lookups appear in Errors.

Start with the default 256 MB run configuration. Duration depends on workflow depth, the number of websites and source response times.

The run summary exposes validDomains, scheduledDomains, skippedByChargeLimit, and details.chargeLimitReached. If a budget limits a comparison or market map, its shares describe the collected group, not the websites that were not processed.

🧮 Tips for large lists

• Paste the whole list. Up to 10,000 websites per run; duplicates, www. prefixes and full URLs are normalised for you.
• Filter before you pay. Set Keep websites with at least or Global rank no worse than so small sites you do not need are never charged.
• Keep the default connection settings and concurrency. Start with 10 websites at a time; raise it only after checking the results for your domain list.
• Set a budget. Maximum charge per run caps the cost; the Actor plans the run around it and reports what it skipped.
• Start with Traffic & audience. It is the fastest workflow. Run Everything we can get only on the websites that passed your first filter.

❓ FAQ

Do old inputs still work?

Yes. base_data, similar_sites, aitdk, and all keep their existing values and legacy nested fields. The new flat fields are additive.

Why is a field empty?

Public coverage varies by domain, size, source, and snapshot. Optional fields remain null, empty arrays, or empty objects instead of being guessed.

What if Similarweb refuses anonymous traffic data?

The Actor uses the current public extension protocol with a stable client identity. An explicit anonymous exceeded / register response is not retried on more paid proxy sessions and is never exported as a website with zero traffic. Other domains can remain available, so that response does not disable the entire extension source. The client identity stays unchanged. The Actor tries the public website over direct HTTP, then Datacenter when available, without Residential transfer or a paid CAPTCHA solver for this fallback.

When the website supplies a reliable profile, baseDataSource: public_website, baseDataStatus: partial, status: partial and dataLimitations identify its coverage. Only the explicitly reported latest month is placed in absolute traffic history; the website's normalized graph is not converted into invented visit counts. Locked channel shares, keyword volumes and AI fields remain unavailable. Full channel concentration, acquisition-diversity and organic/paid aggregate scores also remain unavailable when the channel split is incomplete. Full extension responses keep their original fields and use baseDataSource: extension_api. If neither source supplies traffic, the lookup goes to Errors as source_anonymous_quota_exceeded, without a collection-result charge. In all, independent sources can still produce useful partial profiles with explanatory notes. Custom proxy inputs keep their selected routing and do not enable this direct fallback. Run summary records quota counts and sample domains under details.requestRouting, and fallback activity under details.publicTrafficFallback.

Failed and untracked websites do not consume paid result slots. A maximum charge that covers 100 results allows the Actor to continue through the supplied list until it saves 100 paid profiles, exhausts the input, or encounters a source stop.

Market-map competitor discovery advances with seed-profile processing instead of fetching the whole input far ahead of it. Its initial window also respects the remaining result allowance. Stopping or reaching that allowance closes discovery requests; completed collections retain the same seed and competitor data.

What happens if I enter something incorrectly?

Invalid website entries are skipped with a warning. If none remain, or an input setting is invalid, the Actor exits normally with zero results and an explanation in Run summary (status: invalid_input, message, actionNeeded), without requesting website data. Apify's input form can also reject incorrectly typed settings before a run starts. Diagnostic messages are not inserted into the paid results dataset.

Does a country or keyword view create extra billable results?

No. These views display arrays already stored inside a website result. They do not create new dataset items or trigger extra collection events.

Are failed domains mixed into my export?

No. Valid, partial and untracked results go to the default dataset. Real failures — a source that refused or answered with nothing usable — go to Errors with a short reason such as not_found or source_unavailable; they are not charged as collection results.

I submitted 100 domains and got far fewer rows. Where did the rest go?

They should now all be there. Until September 2026 a domain Similarweb does not measure was dropped silently, so a list of small businesses could finish as a successful run with an empty dataset. Those domains are now returned with trafficDataAvailable: false and cost nothing. If a domain is still missing, it is in the Errors dataset with the reason.

What does in-depth data cost me?

Nothing beyond the Everything we can get result event. Websites with no in-depth profile come back with indepthStatus: "unavailable" and a note, and are charged exactly like any other row.

Why did in-depth data stop partway through my run?

Open Run summary and inspect details.inDepth.stoppedBecause. If the reason is the run's spending limit, increase Maximum charge per run for the next collection. Repeated source checks can reach the collection's attempt limit; retry the remaining websites in a separate run. If a profile is unavailable from the source, review the returned notes and retry that website later. Other collected website data remains available.

Why are some referrers missing from the in-depth lists?

Similarweb withholds part of those lists from anonymous visitors and ships the withheld entries with an empty domain. Exporting them would put blank rows in your spreadsheet, so they are dropped and counted in indepthWithheldRows. The incomingReferrerCount and outgoingReferrerCount totals still reflect the full figure the source reports.

Do I need a Similarweb account or API key?

No. The Actor reads publicly available data and needs only your Apify account.

How fresh is the data?

Every run fetches the latest data the sources publish at that moment. Similarweb updates its figures monthly, so the newest month in trafficHistory and snapshotDate is usually the previous calendar month.

Can I get more than three months of traffic history?

The public source publishes the last three months per run. trafficHistoryMonths reports the number of available months.

That three-month history applies to full extension responses. A partial public-website fallback contains only the explicitly reported latest month and identifies that limit in dataLimitations.

Can I get daily or weekly visits?

No. The source publishes monthly estimates only; running more often returns the same month until Similarweb publishes the next one.

How accurate are the numbers?

They are Similarweb's modelled estimates, not the website's own analytics. They are best used to compare websites and spot trends; small sites carry more uncertainty than large ones.

Is technology detection definitive?

No. It reports signatures present on the public homepage. Server-side and hidden technologies may not be visible.

Which examples can I run?

The input examples use real websites and can be copied into Input → JSON. The result fragments use illustrative values and reserved .test domains to explain the field structure. A live run returns the latest data available from its sources.

✅ Collection and budget handling update

The existing workflows, source fields and result events are retained. Current event prices are unchanged. A matching website still receives the requested available data.

Known domain-ending filters can be checked before collection. In all, known traffic and homepage criteria are checked before unrelated profile enrichment when those filters are active. Missing numeric source values remain unknown, rather than becoming guessed zeros or definite rejections.

Filtered candidates remain free of result-event charges and do not consume the result slots the run's remaining budget can fund. The Actor continues to later matching websites within the input and run limits. OUTPUT.filteredEarly counts definite rejections that avoided later enrichment; filterBreakdown explains their criteria. Platform startup costs can still exist when no website matches.

🧩 More detail, compatible inputs and clearer estimates

Keyword phrases: one through five words 🔎

aitdk and all now include keywordPhrases1Word, keywordPhrases2Words, keywordPhrases3Words, keywordPhrases4Words and keywordPhrases5Words. Each export-ready array contains up to 20 phrases with keyword, count, total, density and wordCount. Density is a percentage of possible contiguous windows of that length in the fetched visible homepage text. keywordDensity.phraseWindowCounts exposes each denominator. The phrase arrays are stored once, rather than duplicated in another large nested object.

Phrases retain internal connecting words and do not join different paragraphs into an invented phrase. Scripts, styles and hidden code blocks are excluded. The HTML page's published language selects the relevant stop-word list when supported; the Unicode tokenizer does not require English text. The stop-word library includes more than 50 languages. This is word/phrase frequency analysis, not semantic language understanding or a dedicated word segmenter for every writing system.

Existing keywordDensity.keywords, totalTokens and pageTitle remain available. Their previous unigram calculations are preserved for existing integrations. New phrase densities use the explicitly documented window denominator; compare like-sized groups. textScope: "fetched_homepage_html" means the fetched homepage HTML, not every page of the website or content rendered only by JavaScript.

The phrase table includes collection status and availability notes. Access-error or verification-only pages are not treated as the website's own keyword content.

{
  "domains": ["apify.com", "python.org"],
  "mode": "aitdk"
}

Sampled keyword and channel opportunities ⚔️

compare adds keywordGapVsPrimary: terms found in another site's published top-keyword sample but absent from the first domain's available sample, with published volume and CPC when present. keywordGapStatus distinguishes a valid sample from missing input data. This is not an exhaustive search-keyword inventory and an absent sampled term does not prove the primary site never ranks for it.

channelGapsVsPrimary compares organic search, paid search, referrals and AI-referral shares in percentage points. It is populated only when both sites publish the same snapshot date; inspect benchmarkPeriodsMatch. Existing comparison scores, traffic ranks and gap fields remain intact.

{
  "domains": ["ahrefs.com", "semrush.com", "apify.com"],
  "mode": "compare"
}

Country and channel visit estimates 🌍

Traffic-bearing rows add countryTrafficEstimates and channelTrafficEstimates. These explicitly labelled calculations multiply the collected monthly visit estimate by the published country/channel share. They are not separately observed visitor counters. trafficEstimatesMethod records the formula and trafficEstimatesMonth records the underlying month when available. Unknown shares are omitted; a source-published zero is kept. trafficHistoryMonths tells you how many actual monthly data points are available.

More published domain and profile details 🏢

WHOIS output can include registrarIanaId, registrantCountry, abuseEmails and abusePhones when the public RDAP record provides them. Registry abuse contacts remain inside whois; they are not represented as the website company's sales contacts.

With mode: "all" and includeInDepth: true, the profile can also supply the headquarters state, competitor/referrer icons, search-only organic/paid shares and the profile's own keyword sample. organicSearchShare/paidSearchShare describe the split within search, while the existing acquisition-channel shares describe total site traffic. These are different denominators.

Locked referral/advertising entries are counted separately as incomingReferrersWithheldCount, outgoingReferrersWithheldCount and adsPublishersWithheldCount. referralDataCoverage makes partial coverage explicit. The Actor retains available rows; it does not invent identities for locked entries. All existing audience, ranking, company, contact and technology fields remain available under their source-dependent conditions.

Compatible website URLs and API aliases 🔗

You may pass a bare domain, an ordinary website URL, or a Similarweb profile URL such as https://www.similarweb.com/website/ahrefs.com/. A Similarweb profile resolves to the analysed website rather than analysing similarweb.com itself.

For migrating API workflows, websites is accepted as an alias for domains; datasetMode and searchType can select an omitted mode. searchType: "similarweb" means base_data. Explicit conflicting workflow names are rejected before collection, so an alias cannot silently upgrade a run to a more expensive workflow. Omitting all mode selectors continues to use base_data.

Open Keyword phrases, Traffic estimates or Keyword & channel gaps in Output to inspect the new fields. These additions use the existing collection events; no new result events were introduced.

Collection reliability and complete results 🔎

The available fields and six workflows are retained. Every collection still requests the current source response; website results are not reused from a previous run. A recovered extended profile is delivered as a partial result when the other sources are unavailable, rather than being discarded. Empty or metadata-only responses are not turned into paid results.

Run diagnostics in OUTPUT include the selected memory allocation and a source-attempt summary to help explain slow or partial collections. Existing prices and chargeable event names are unchanged.

An extended response containing only a zero-filled profile skeleton is marked indepthStatus: "limited" and indepthDataCoverage: "empty_source_payload". Source-provided values are retained, with a note explaining the limitation. rowsWithProfile counts profiles containing source data; profiles counts received profile payloads, including limited ones. A real zero within an otherwise populated profile remains a zero.

🛑 Completion and empty results

Each run performs a finite collection. Source tasks are cancelled and awaited when their parent fails, the result-charge limit is reached, or collection is cancelled. Network sessions close before the run finishes, including error paths. Optional access-session cache publication has one five-second allowance after network cleanup, regardless of the number of sessions.

Source errors, filter rejections and untracked-domain diagnostics do not receive a result-event charge. A run can still incur platform startup, source-request or solver costs while establishing that no data is available.

An HTTP response from the asynchronous /runs API confirms that the run was started; check the run's final status to confirm completion. A client-side HTTP timeout or disconnected integration is not a cancellation request. Use the run status and the platform's abort action when an integration needs to stop unfinished work.

Collection also ends after a sustained sequence of source responses without useful data, or when sources stop making progress. A definite filter rejection is tracked separately from an empty source response. Missing optional values do not make an otherwise useful profile empty; real zero and false values are preserved. Rows containing only an identity and empty metadata are not sold as successful profiles. OUTPUT.details.resourceGuard reports these collection stops, and PROGRESS provides live counts.

🎯 Filter combinations and small run budgets

The Actor checks whether selected include/exclude lists can possibly match before collection. For example, allowing only .com while excluding .com returns invalid_input with a correction message. Partly overlapping lists remain valid, and settings that do not apply to the selected workflow remain inactive with a warning.

In aitdk, domain-age criteria can reject a website before its homepage is collected; contact and technology criteria can reject a homepage before unrelated domain details are requested. Matching websites retain the full requested available fields. In all, these checks also precede unrelated enrichment. A small remaining result budget reduces how many complete websites start at once; rejected candidates leave room for later matches.

market_map continues to later matching websites when earlier candidates are filtered out. Market shares and comparison ranks still use the collected cohort before output filters are applied. These cohort workflows may therefore inspect websites that are not ultimately saved.

OUTPUT.datasetRows includes paid results and free untracked-domain responses; OUTPUT.results continues to count paid result rows. notTracked, filteredOut and billing explain the difference. The effective event prices used by the run are included in the summary. Filtering removes result-event charges; it does not make the work needed to evaluate a website cost-free. Existing result prices are unchanged.

⚡ Live profiles and complete cohort analytics

Ready website profiles now appear in Results immediately in every workflow, including Compare websites and Map a market. A slower website does not hold back profiles that have already finished. Each successful profile receives its existing result-event charge once.

For compare, open Final comparison for the full source fields together with compareRank, traffic shares, gaps, scores and keyword opportunities. For market_map, open Final market analysis for the full source fields together with market ranks, shares and relationships. These calculations require the collected group, so they are published after collection; they do not create another result-event charge. The default dataset contains the streamed profiles, and analysisDatasetId identifies the separate final dataset for API integrations. Use that dataset when exporting the calculated comparison or market columns.

cohortComplete, cohortSitesCollected, cohortSitesRequested and cohortStopReason describe the scope of the final calculations. A collection that stops at a budget, an empty-source stop or cancellation keeps its already delivered profiles; available final calculations are explicitly marked as a partial cohort. They are not represented as a complete market survey.