SimilarWeb Website Traffic Scraper

Hand it a list of domains. Get back a full competitive profile for each one — traffic, three rankings with history, channel mix, geography, keywords, audience, tech and competitors. 56 fields per domain, as JSON, CSV or Excel.

Three things that are different here

1. You pay only for records that contain data. A domain that comes back blocked, failed, uncovered or empty is counted in the run report and then dropped. It is never written to your dataset and never billed. Check the other traffic scrapers on this Store: you will not find this promise on any of them, and you will find their users saying so in the issues.

2. Every record tells you how good it is. snapshotDate gives the month the data covers. isDataFromGoogleAnalytics tells you whether the numbers are the site's own verified analytics or an estimate. lockedSections names exactly which sections SimilarWeb is hiding behind its own paywall for that domain. filledFieldCount, httpStatus, fetchedAt and sourceUrl let you trace and rank any row. You never have to wonder whether a blank means zero, missing, or paywalled.

3. It reads the page everyone can see. Most scrapers in this category call SimilarWeb's internal data endpoint. That endpoint fails often, and a failed call still costs you a result when the Actor writes an empty row. This one parses the public profile page instead. Ten out of ten domains returned a complete record in testing, with no retries.

What you get

Traffic and engagement Monthly visits, month-over-month change, bounce rate, pages per visit, average visit duration.

Three rankings, with history Global, country and category rank — each with the previous month's value, the change, and a three-month history series. Most traffic scrapers give you today's number and nothing to compare it against.

Where the traffic comes from Ten traffic-source channels with their share (direct, organic, referral, social, paid, mail, affiliate and more), top countries with share and share change, top referring sites and top outgoing sites.

Search Organic vs paid split, total keyword count, and the top keywords with their cost per click.

Audience and competition Age distribution, male/female split, similar sites with an affinity score, and the sites ranked immediately around this domain.

Technology and advertising Count of detected technologies and technology categories, advertising networks and advertising sites.

Typical uses

Competitive research before entering a market. Qualifying a partner or an acquisition target. Tracking whether a competitor's traffic is rising or falling month over month. Enriching a list of domains with traffic size. Checking which channels a site actually depends on before copying its strategy.

Fields worth knowing about

snapshotDate tells you which month the data covers. Traffic estimates are monthly, and a number without its month is not comparable to anything. Not every scraper reports this.

isDataFromGoogleAnalytics tells you whether the numbers come from the site's own verified analytics or from estimation. When a site owner connects their analytics, the figures stop being an estimate. If you are making a decision on this data, you want to know which kind you are looking at.

lockedSections names the sections SimilarWeb keeps behind its own paywall for that specific domain — typically referrals, outgoing traffic or advertising. When a section is locked, the public page simply shows less, and a scraper that does not tell you this leaves you guessing whether the data is missing or genuinely zero. isPartial is the one-line version of the same answer.

sourceUrl, fetchedAt, httpStatus and filledFieldCount travel with every record, so a row can be traced back to the exact page and moment it came from, and you can sort a batch by how complete each record is.

Example record (trimmed)

{
  "domain": "github.com",
  "snapshotDate": "2026-08-01T00:00:00+00:00",
  "globalRank": 50,
  "globalRankPrevious": 49,
  "globalRankChange": -1,
  "countryCode": "US",
  "countryRank": 81,
  "categoryRank": 4,
  "categoryId": "computers_electronics_and_technology/programming_and_developer_software",
  "visitsTotal": 649321442,
  "visitsChange": 0.0179,
  "bounceRate": 0.3666,
  "pagesPerVisit": 5.77,
  "avgVisitDuration": "00:06:24",
  "isDataFromGoogleAnalytics": false,
  "trafficSources": [{ "source": "direct", "rank": 1, "share": 0.526 }],
  "topCountries": [{ "countryCode": "US", "share": 0.186, "shareChange": -0.026 }],
  "topKeywords": [{ "keyword": "github", "cpc": 1.63 }],
  "similarCompetitors": [{ "domain": "stackoverflow.com", "affinity": 1, "categoryRank": 81 }],
  "globalRankHistory": [{ "date": "2026-06-01T00:00:00+00:00", "rank": 47 }],
  "lockedSections": ["incomingReferrals", "outgoingReferrals", "ads"],
  "isPartial": true,
  "filledFieldCount": 54,
  "httpStatus": 200,
  "fetchedAt": "2026-09-16T17:20:32.878Z",
  "sourceUrl": "https://www.similarweb.com/website/github.com/"
}

Input

Paste domains one per line. Bare domains and full URLs both work — github.com, www.github.com, https://github.com/features all resolve to the same record. Duplicates are removed.

maxItems caps how many domains are processed so a long list cannot produce an unexpected charge. concurrency controls how many run in parallel.

Failures are loud

The run fails with an explanation rather than finishing quietly with less data than you expected. If the page structure changes, you get a parse error naming the affected domains, not an empty dataset. Counters are reconciled at the end of every run: requested, written, robots-blocked, HTTP-failed, parse-failed. The full breakdown is written to the RUN_STATS key.

Transient gateway errors are retried up to three times with a fresh proxy session before a domain is given up on.

Limits and compliance

robots.txt is checked before every request and a disallowed path is never fetched — the run reports it and skips, it does not force its way through.

No personal data is collected. The output is built from an explicit list of known fields, so nothing outside that list can leak into your dataset.

Data is a monthly estimate from a public page. Very small sites often have no traffic data at all; those domains are counted and reported rather than written as empty records.

Pricing

$6.00 per 1,000 records, everything included. No start fee, no minimum charge per run, and no separate platform usage line on your bill. The number on this page is the number you pay.

Paid Apify plans pay less, automatically — there is nothing to apply for: Bronze $5.40, Silver $4.80, Gold and above $4.20 per 1,000 records. The Free plan price is unchanged at $6.00.

You are charged per record written, and a record is only written when it contains data. Domains that are skipped by robots.txt, fail after three retries, or have no traffic data at all are reported in the run statistics and cost you nothing.

Measured performance

Run on 16 September 2026 against a deliberately hostile list of 150 domains: 30 large sites, 40 mid-size SaaS products, 30 small tools, 20 very low traffic sites, and 30 domains that do not exist at all.

Records written and charged	115
Domains reported and not charged	35
Non-existent domains that produced a charge	0
Parse failures	0
Paths blocked by robots.txt and skipped	0
Median fields filled per record	54 of 56
Records carrying traffic figures	115 of 115
Wall clock	15 minutes at concurrency 8

Every one of the 30 non-existent domains was counted in the run report and dropped. None of them reached your dataset and none of them reached your bill. Four real domains returned gateway errors after three retries each and were also reported rather than written.