SimilarWeb Scraper — Website Traffic, Rankings, WHOIS, and Keyword Density

🦅 More from Raven ►	Ahrefs SEO Tools	Semrush Scraper	Similarweb Scraper

Extract website analytics from three data sources: SimilarWeb traffic data, similar sites discovery, and AITDK domain intelligence. Choose one source per run via the Search Type dropdown. Analyze up to 50 domains in parallel.

Pricing

Plan	Price	Best For	Link
Pay per result	$2 / 1,000 results	One-time analysis, small batches	SimilarWeb Scraper
Monthly subscription	$20 / month	Regular monitoring, unlimited usage	SimilarWeb Website Analytics

Both plans have identical functionality — choose the one that fits your workflow.

Search Types

Select one of three modes in the Search Type dropdown:

Search Type	Description	Output Fields
SimilarWeb	Traffic data, rankings, engagement metrics, traffic sources	SiteName, Title, Category, GlobalRank, CountryRank, Engagments, EstimatedMonthlyVisits, TrafficSources, TopCountryShares
Similar Sites	Competitor and alternative websites	SiteName, SimilarSites (Site, TotalVisits, CategoryRank, TopCountryRank, Description)
AITDK	WHOIS domain data and keyword density analysis	SiteName, Whois (registrationDate, expirationDate, registrar, nameservers, status), KeywordDensity (1-5 word phrases)

Key Features

• Bulk domain analysis — submit up to 50 domains at once, all processed simultaneously
• 3 data sources — choose SimilarWeb traffic data, similar sites, or AITDK domain intelligence per run
• Traffic estimates — monthly visits, bounce rate, pages per visit, average session duration, and 3-month visit history (SimilarWeb mode)
• Global and country rankings — worldwide rank, country-specific rank, and industry category rank (SimilarWeb mode)
• Traffic source breakdown — percentage split across direct, organic search, paid search, social, referral, email, and display ads (SimilarWeb mode)
• Geographic distribution — top 5 countries driving traffic with share percentages (SimilarWeb mode)
• Similar sites discovery — related websites with their own traffic data, category rank, and descriptions (Similar Sites mode)
• WHOIS data — domain registration date, expiration date, registrar, nameservers, registrant country, and domain status (AITDK mode)
• Keyword density analysis — 1-word through 5-word phrase frequency analysis with stop word filtering, 50 languages (AITDK mode)
• Flexible proxy support — use Apify Proxy (Residential or Datacenter) or bring your own HTTP proxies
• Error resilience — failed items are logged with error details, other domains still process normally

What Data You Get

SimilarWeb mode

Field	Description
SiteName	Domain name
Title	Website title
Category	Industry category (e.g. computers_electronics_and_technology/search_engines)
GlobalRank	Worldwide traffic rank
CountryRank	Rank within the primary traffic country
Engagments	Total visits, bounce rate, pages per visit, session duration
EstimatedMonthlyVisits	Last 3 months of visit data (raw numbers and formatted)
TrafficSources	Percentage split by channel: Direct, Search, Social, Referrals, Mail, Paid
TopCountryShares	Top 5 countries with traffic share percentages

Similar Sites mode

Field	Description
SiteName	Domain name
SimilarSites	Array of related websites with Site, TotalVisits, CategoryRank, TopCountryRank, Description

AITDK mode

Field	Description
SiteName	Domain name
Whois.registrationDate	Domain registration date
Whois.expirationDate	Domain expiration date
Whois.lastUpdated	Last WHOIS record update
Whois.registrar	Domain registrar name
Whois.registrantCountry	Registrant country code
Whois.nameservers	List of DNS nameservers
Whois.status	Domain status flags (e.g. client transfer prohibited)
KeywordDensity.1word	Single-word frequency (stop words filtered)
KeywordDensity.2words	2-word phrase frequency
KeywordDensity.3words	3-word phrase frequency
KeywordDensity.4words	4-word phrase frequency
KeywordDensity.5words	5-word phrase frequency

Each keyword density entry contains: keyword, count, total (total words on page), density (percentage). Sorted by density descending.

Use Cases

• Competitive analysis — track competitors' traffic trends and traffic sources to spot gaps and opportunities (SimilarWeb mode)
• On-page SEO audit — use keyword density analysis to check keyword stuffing, content relevance, and phrase optimization across 1-5 word n-grams (AITDK mode)
• Domain research — check domain age, expiration dates, registrar history, and DNS configuration via WHOIS data (AITDK mode)
• Marketing budget allocation — understand where competitors get traffic (search, social, direct, paid) and optimize your channel mix (SimilarWeb mode)
• Lead generation and sales intelligence — enrich your CRM with website traffic data to qualify leads and prioritize outreach (SimilarWeb mode)
• Investment due diligence — evaluate company digital performance using traffic trends, monthly visit history, domain age, and market positioning
• Partnership discovery — find similar websites and potential partners in your niche (Similar Sites mode)

How It Works

1. Select search type — choose SimilarWeb, Similar Sites, or AITDK from the dropdown
2. Enter domains — provide up to 50 domains (e.g. google.com, shopify.com)
3. Configure proxy — select Residential proxy for best success rate, or use your own
4. Actor fetches data — all domains processed in parallel using the selected data source
5. Download results — structured JSON output in the Apify dataset, ready to export as JSON, CSV, or Excel

Input Parameters

Parameter	Type	Required	Description
searchType	String	Yes	Data source: similarweb (traffic, rankings), similar_sites (competitors), or aitdk (WHOIS, keyword density)
domains	String[]	Yes	List of domains to analyze, max 50 per run (e.g. google.com, facebook.com)
proxyConfiguration	Object	No	Proxy settings — Apify Proxy groups (RESIDENTIAL, DATACENTER) or custom proxy URLs

Input examples

SimilarWeb mode (traffic data):

{
    "searchType": "similarweb",
    "domains": ["google.com", "facebook.com", "shopify.com"]
}

Similar Sites mode:

{
    "searchType": "similar_sites",
    "domains": ["google.com"]
}

AITDK mode (WHOIS + keyword density):

{
    "searchType": "aitdk",
    "domains": ["google.com"]
}

With custom proxies:

{
    "searchType": "similarweb",
    "domains": ["google.com"],
    "proxyConfiguration": {
        "useApifyProxy": false,
        "proxyUrls": [
            "http://user:pass@proxy-provider.com:port"
        ]
    }
}

Output Examples

SimilarWeb mode

{
    "SiteName": "google.com",
    "Title": "Google",
    "Category": "computers_electronics_and_technology/search_engines",
    "GlobalRank": { "Rank": 1 },
    "CountryRank": { "Country": 840, "CountryCode": "US", "Rank": 1 },
    "Engagments": {
        "Visits": 85756574615,
        "VisitsFormatted": "85.76B",
        "BounceRate": 28.36,
        "PagePerVisit": 8.52,
        "TimeOnSite": 610
    },
    "EstimatedMonthlyVisits": {
        "2025-11-01": 82284033110,
        "2025-12-01": 84172772881,
        "2026-01-01": 85756574615
    },
    "TrafficSources": {
        "Direct": 85.83,
        "Search": 8.35,
        "Referrals": 4.46,
        "Social": 0.76,
        "Paid Referrals": 0.33,
        "Mail": 0.28
    },
    "TopCountryShares": [
        { "CountryCode": "US", "Value": 24.59 },
        { "CountryCode": "IN", "Value": 5.55 },
        { "CountryCode": "JP", "Value": 5.47 }
    ]
}

Similar Sites mode

{
    "SiteName": "google.com",
    "SimilarSites": [
        { "Site": "bing.com", "TotalVisits": 1234567890, "CategoryRank": 2, "TopCountryRank": 3, "Description": "..." },
        { "Site": "yahoo.com", "TotalVisits": 987654321, "CategoryRank": 5, "TopCountryRank": 8, "Description": "..." }
    ]
}

AITDK mode

{
    "SiteName": "google.com",
    "Whois": {
        "registrationDate": "1997-09-15T04:00:00Z",
        "expirationDate": "2028-09-14T04:00:00Z",
        "lastUpdated": "2019-09-09T15:39:04Z",
        "registrar": "MarkMonitor Inc.",
        "registrantCountry": "US",
        "nameservers": ["NS1.GOOGLE.COM", "NS2.GOOGLE.COM", "NS3.GOOGLE.COM", "NS4.GOOGLE.COM"],
        "status": ["client delete prohibited", "client transfer prohibited", "client update prohibited"]
    },
    "KeywordDensity": {
        "1word": [
            { "keyword": "google", "count": 5, "total": 366, "density": 1.37 },
            { "keyword": "english", "count": 5, "total": 366, "density": 1.37 }
        ],
        "2words": [
            { "keyword": "cookies und", "count": 3, "total": 366, "density": 0.82 }
        ],
        "3words": [
            { "keyword": "cookies und daten", "count": 3, "total": 366, "density": 0.82 }
        ],
        "4words": [
            { "keyword": "verwenden wir cookies und", "count": 3, "total": 366, "density": 0.82 }
        ],
        "5words": [
            { "keyword": "verwenden wir cookies und daten", "count": 2, "total": 366, "density": 0.55 }
        ]
    }
}

Proxy and Troubleshooting

Residential proxies are recommended for the best success rate. SimilarWeb actively blocks datacenter IPs.

Symptom	Likely Cause	Fix
Most domains return 403/429	IP is blocked	Switch to RESIDENTIAL proxies
Single-domain failure	Domain has no SimilarWeb data	Check domain name; expected behavior
Timeouts on all domains	Proxy is down	Verify proxy URLs or use Apify Proxy
Works for ~10 domains, then fails	IP not rotating	Use per-request rotation, avoid sticky sessions
WHOIS data missing	Rate limit on WHOIS lookup	Normal for high-volume runs, other data still present

FAQ

What are the search types? There are 3 modes: SimilarWeb returns traffic and ranking data, Similar Sites returns competitor websites, AITDK returns WHOIS domain info and keyword density analysis. Select one per run.

How many domains can I analyze per run? Up to 50 domains in a single run, all processed in parallel. Typical response time is 1–10 seconds per domain.

Does the scraper work with subdomains? Yes. Pass subdomains like translate.google.com or maps.google.com and get separate analytics for each.

What proxy should I use? Residential proxies give the highest success rate. Datacenter proxies are faster and cheaper but get blocked more often. Start with RESIDENTIAL if you need reliability.

Can I use my own proxies instead of Apify Proxy? Yes. Select "Custom proxies" in the proxy configuration and add your proxy URLs. Make sure they support per-request rotation.

What happens if a domain fails? The Actor continues processing all remaining domains. Failed items include an error field with details.

How does keyword density work? In AITDK mode, the Actor fetches the page HTML, strips non-visible elements (scripts, styles, iframes, etc.), tokenizes the visible text, and calculates n-gram frequencies for 1-5 word phrases. Stop words from 50 languages are filtered from single-word results.

What's the difference between the monthly and pay-per-use plans? Identical functionality. Monthly ($20/mo) is better for ongoing monitoring; pay-per-use ($2/1k results) is better for occasional analysis.

Having issues? Help me fix them faster

If you experience any problems, please share your run data with me so I can debug and improve the Actor:

1. Go to Apify Security Settings
2. Find "Share run data with developers"
3. In the "Manage list of Actors" section, check this Actor (or All Actors)
4. Save

This data is used only for debugging and helps me resolve issues much faster. Thank you!

Support

Questions, bug reports, or feature requests: afrcanec@gmail.com

🏷️ Tags

SimilarWeb Scraper, Website Traffic Analyzer, SimilarWeb API, Website Analytics Extractor, Global Ranking Checker, Traffic Source Breakdown, Organic Keywords Extractor, AI Traffic Tracker, Domain WHOIS Lookup, Keyword Density Analyzer, Competitor Traffic Analysis, Website Ranking Data, SEO Analytics Tool, SimilarWeb Data Extractor, Digital Marketing Intelligence, Monthly Visits Tracker, Bounce Rate Analyzer, Similar Sites Finder, How to Scrape SimilarWeb, Extract Website Traffic Data, Domain Research Tool, Market Share Analysis, Website Comparison Tool, Referral Traffic Analytics, Country Traffic Distribution, Web Analytics Scraper, Investment Due Diligence, Lead Qualification Data, Website Performance Metrics, AI Referral Traffic Monitor