# Semrush Website Traffic Scraper (`axlymxp/semrush-website-traffic-scraper`) Actor

Get Semrush website analytics in bulk without a Semrush login. Export country and category rank, organic and paid search traffic, desktop/mobile visit split, plus 6 months of backlink and referring-domain history for any domain, or auto-discover trending sites by category. Pay only for results.

- **URL**: https://apify.com/axlymxp/semrush-website-traffic-scraper.md
- **Developed by:** [axly](https://apify.com/axlymxp) (community)
- **Categories:** SEO tools, Marketing, Business
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.00 / 1,000 dataset items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Semrush Website Traffic Scraper

Pull **Semrush website analytics in bulk** — country and category rank, organic and paid search
traffic, desktop/mobile split, and **6 months of backlink and referring-domain history** — for any
list of domains. No Semrush account, no login, no CAPTCHA, no proxy required.

Point it at a list of domains, or let it **discover trending websites** from Semrush's own rankings
across 111 industry categories and export the winners automatically.

### Who it's for

- **SEO agencies & consultants** — qualify inbound leads and build competitive pitch decks with real
  rank, traffic and backlink numbers instead of screenshots.
- **Growth & marketing teams** — track the traffic trajectory of your competitors month over month.
- **Investors & analysts** — size a market or diligence a company from traffic trend data.
- **Lead generation** — score thousands of prospect domains by traffic and authority, then filter to
  the ones worth contacting.
- **Data teams & researchers** — build a longitudinal traffic dataset by scheduling monthly runs.

### What makes it different

Most Semrush actors need a paid Semrush login (and often a pool of accounts plus residential
proxies) to work — which is exactly why they break. This actor reads **only Semrush's public,
server-rendered website overview pages**, so there is nothing to log into and nothing to get banned.
That includes the **full backlink and referring-domain history**, which the public pages expose as
monthly series.

Metrics that Semrush genuinely keeps behind its paywall (keyword lists, competitor sets, site audit,
traffic-by-country, traffic journey) are returned as `null` rather than guessed or faked.

### Output fields

Each row is one domain, flattened and ready for a spreadsheet or database.

#### Identity & categorization

| Field | Type | Description |
| --- | --- | --- |
| `domain` | string | Root domain analyzed, e.g. `stripe.com`. |
| `url` | string | Semrush overview page the data came from. |
| `brand_name` | string | null | Brand name Semrush associates with the domain. |
| `country` | string | null | ISO code of the domain's primary country. |
| `categories` | array | Semrush industry category names. |
| `category_slugs` | array | URL slugs of the category names. |
| `favicon` | string | null | Favicon image URL. |
| `data_date` | string | null | Date the metrics refer to (Semrush lags ~1 month). |
| `date_modified` | string | null | Date Semrush last refreshed the page. |

#### Rankings

| Field | Type | Description |
| --- | --- | --- |
| `global_rank` | integer | null | Global traffic rank (often null on the public page). |
| `global_rank_trend` | string | null | `up`, `flat` or `down`. |
| `country_rank` | integer | null | Traffic rank within the domain's country. |
| `country_rank_country` | string | null | Country the rank applies to. |
| `country_rank_trend` | string | null | `up`, `flat` or `down`. |
| `category_rank` | integer | null | Rank within the primary industry category. |
| `category_rank_category` | string | null | Category the rank applies to. |
| `category_rank_trend` | string | null | `up`, `flat` or `down`. |

#### Search traffic

| Field | Type | Description |
| --- | --- | --- |
| `search_traffic_date` | string | null | Date of the search-traffic figures. |
| `organic_traffic` | number | null | Estimated monthly organic search visits. |
| `organic_traffic_change_percent` | number | null | Month-over-month change in organic traffic (%). |
| `paid_traffic` | number | null | Estimated monthly paid search visits. |
| `paid_traffic_change_percent` | number | null | Month-over-month change in paid traffic (%). |
| `total_search_traffic` | number | null | Organic plus paid search visits. |

#### Visits & device split

| Field | Type | Description |
| --- | --- | --- |
| `traffic_date` | string | null | Date of the visits/device figures. |
| `visits` | number | null | Estimated total monthly visits. |
| `visits_change_percent` | number | null | Month-over-month change in visits (%). |
| `desktop_visits` | number | null | Monthly visits from desktop. |
| `mobile_visits` | number | null | Monthly visits from mobile. |
| `desktop_share_percent` | number | null | Desktop share of visits (%). |
| `mobile_share_percent` | number | null | Mobile share of visits (%). |

#### Backlinks & authority

| Field | Type | Description |
| --- | --- | --- |
| `backlinks` | number | null | Total backlinks in the most recent month. |
| `backlinks_previous_month` | number | null | Total backlinks in the prior month. |
| `backlinks_change_percent` | number | null | Month-over-month change in backlinks (%). |
| `referring_domains` | number | null | Unique referring domains in the most recent month. |
| `referring_domains_previous_month` | number | null | Unique referring domains in the prior month. |
| `referring_domains_change_percent` | number | null | Month-over-month change in referring domains (%). |
| `google_penalty_risk` | string | null | Semrush's Google penalty risk label, e.g. `Low`. |

#### History arrays (set `includeHistory` to `true`)

| Field | Type | Description |
| --- | --- | --- |
| `search_traffic_history` | array | Monthly organic/paid/total search traffic, most recent first. |
| `traffic_by_device_history` | array | Monthly visits split into desktop and mobile, most recent first. |
| `visits_history` | array | Monthly total visits, most recent first. |
| `backlinks_history` | array | Monthly backlink counts, most recent first. |
| `referring_domains_history` | array | Monthly referring-domain counts, most recent first. |

#### Trending context (trending mode only)

| Field | Type | Description |
| --- | --- | --- |
| `trending_rank` | integer | null | Position in the Semrush trending ranking. |
| `trending_country` | string | null | Country of the trending ranking. |
| `trending_category` | string | null | Category slug of the trending ranking. |
| `trending_total_traffic` | string | null | Traffic as displayed in the trending table, e.g. `24.54B`. |
| `trending_desktop_share_percent` | number | null | Desktop share shown in the trending table. |
| `trending_mobile_share_percent` | number | null | Mobile share shown in the trending table. |
| `trending_mom_percent` | number | null | Month-over-month traffic change from the trending table. |
| `trending_yoy_percent` | number | null | Year-over-year traffic change from the trending table. |
| `trending_top_source` | string | null | Largest traffic source, e.g. `Direct`. |

#### Run metadata

| Field | Type | Description |
| --- | --- | --- |
| `source_mode` | string | Discovery mode that produced the row (`domains` or `trending`). |
| `found` | boolean | `false` when Semrush had no overview data for the domain. |
| `scraped_at` | string | UTC timestamp when the row was scraped (ISO-8601). |

### Use cases

**Rank a lead list by traffic before you call anyone.** Feed 5,000 prospect domains in `domains`
mode, sort the dataset by `visits` and `backlinks`, and route only the top slice to sales.

**Build a competitor traffic tracker.** Schedule a monthly run over your competitor set and append
to the same dataset — `visits_change_percent`, `organic_traffic_change_percent` and
`backlinks_change_percent` give you the deltas without any extra maths.

**Find category winners you've never heard of.** Turn on `allCategories` for a country and get the
top 20 trending sites in each of 111 categories, enriched with full overview metrics.

**Diligence a target company.** `backlinks_history` and `visits_history` show whether growth is real
and sustained or a one-month spike.

**Spot link-building momentum.** `referring_domains_change_percent` reveals who is actively
acquiring links right now — useful for both competitive intel and outreach targeting.

### Input parameters

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `mode` | string | `domains` | `domains` analyzes the list you provide; `trending` discovers domains from Semrush's trending rankings. |
| `domains` | array | — | Domains or URLs to analyze, e.g. `stripe.com` or `https://www.nytimes.com/section/world`. URLs are normalized to the root domain automatically. |
| `country` | string | `us` | Country code for the trending ranking, e.g. `us`, `gb`, `de`, `in`. |
| `category` | string | `all` | Trending category slug, e.g. `finance`, `travel-and-tourism`, `mass-media`. An unknown slug fails fast and lists the valid options. |
| `allCategories` | boolean | `false` | Sweep every trending category for the chosen country (111 categories × 20 rows). |
| `includeHistory` | boolean | `true` | Include the 6-month history arrays. Turn off for a compact, flat CSV. |
| `maxItems` | integer | `100` | Stop after this many rows. |
| `minRequestInterval` | number | `0.5` | Politeness delay in seconds before each domain lookup. |
| `proxyConfiguration` | object | disabled | Optional. Not required — the public pages are not proxy-gated. |

#### Example input — domain list

```json
{
    "mode": "domains",
    "domains": ["stripe.com", "shopify.com", "https://www.nytimes.com/section/world"],
    "includeHistory": true,
    "maxItems": 100
}
```

#### Example input — trending discovery

```json
{
    "mode": "trending",
    "country": "us",
    "category": "finance",
    "includeHistory": false,
    "maxItems": 20
}
```

#### Example output row (truncated)

```json
{
    "domain": "stripe.com",
    "url": "https://www.semrush.com/website/stripe.com/overview/",
    "country": "US",
    "categories": ["Computer Software and Development", "Finance"],
    "data_date": "2026-08-15",
    "country_rank": 161,
    "country_rank_country": "United States",
    "country_rank_trend": "down",
    "category_rank": 6,
    "category_rank_category": "Finance",
    "organic_traffic": 5667468,
    "organic_traffic_change_percent": 2.48,
    "paid_traffic": 439916,
    "visits": 203227314,
    "visits_change_percent": 12.94,
    "desktop_share_percent": 51.25,
    "mobile_share_percent": 48.75,
    "backlinks": 151822916,
    "backlinks_previous_month": 167449561,
    "backlinks_change_percent": -9.33,
    "referring_domains": 664614,
    "referring_domains_change_percent": 4.86,
    "google_penalty_risk": "Low",
    "backlinks_history": [
        { "date": "2026-07-01", "backlinks": 151822916 },
        { "date": "2026-06-01", "backlinks": 167449561 }
    ],
    "found": true,
    "scraped_at": "2026-09-17T02:12:00Z"
}
```

### Scheduling & webhooks

Semrush refreshes these pages roughly monthly, so a **monthly schedule** is the natural cadence for
trend tracking. In the Apify Console open **Schedules → Create new**, pick this actor, and save your
input.

Add a **webhook** (Actor → Integrations) on `ACTOR.RUN.SUCCEEDED` to push each finished dataset
straight into Google Sheets, Slack, Zapier, Make, or your own endpoint. Runs resume from a
checkpoint if they are aborted and resurrected, so long sweeps pick up where they stopped instead of
starting over.

### Using it from an AI assistant (MCP)

This actor works as a tool for LLM agents through the **Apify MCP server**, so an assistant can pull
live traffic data mid-conversation:

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": ["-y", "@apify/actors-mcp-server"],
            "env": { "APIFY_TOKEN": "<YOUR_APIFY_TOKEN>" }
        }
    }
}
```

Then ask things like *"Compare the organic traffic and backlink growth of stripe.com and
adyen.com over the last six months."*

You can also call it from code with the Apify API or the JS/Python clients — every run's output is a
standard dataset available as JSON, CSV, Excel or XML.

### FAQ

**Do I need a Semrush subscription or login?**
No. The actor only reads Semrush's public website overview and trending pages. There is no account,
API key, or cookie to supply.

**Do I need proxies?**
No. The pages are not proxy-gated. A `proxyConfiguration` input exists if you want to route traffic
through Apify Proxy anyway, but leaving it off is fine and cheaper.

**Why are `authority_score`, keywords, or competitor lists missing?**
Those are genuinely paywalled by Semrush and are not present in the public page data. This actor
returns `null` for what it cannot see rather than inventing values. Backlinks and referring domains
**are** available, including their monthly history.

**How fresh is the data?**
Semrush's public overview data lags real time by roughly one month. Every row carries `data_date`
and `date_modified` so you always know the vintage.

**How accurate is it?**
The numbers are exactly what Semrush publishes — this actor does no estimation of its own. Semrush's
figures are themselves modelled estimates, so treat them as directional, as you would in the Semrush
UI.

**What happens with a domain Semrush doesn't track?**
The run continues. In `domains` mode the domain is skipped and logged; in `trending` mode the row is
still emitted with `found: false` so you keep the ranking context.

**Can it handle thousands of domains?**
Yes. Raise `maxItems`, and the run checkpoints its progress so an aborted-and-resurrected run
continues rather than restarting.

**Is scraping this legal?**
The actor reads only publicly accessible pages — no login is bypassed and no paywalled content is
unlocked. You are responsible for using the output in line with Semrush's terms and the laws that
apply to you.

**Something broke — how do I get help?**
Open an issue on the actor's **Issues** tab with your input JSON and the run ID.

# Actor input Schema

## `mode` (type: `string`):

How to choose which domains to analyze. 'Domain list' analyzes the domains you provide. 'Trending websites' pulls Semrush's top-20 ranking for a country/category and then enriches each domain.

## `domains` (type: `array`):

Domains or URLs to analyze, e.g. "stripe.com" or "https://www.nytimes.com/section/world". URLs are reduced to their root domain automatically. Required when mode is 'Domain list'.

## `country` (type: `string`):

Two-letter country code for the trending ranking, e.g. 'us', 'gb', 'de', 'in'.

## `category` (type: `string`):

Trending category slug, e.g. 'all', 'finance', 'travel-and-tourism', 'mass-media', 'computer-software-and-development'. Use 'all' for the overall ranking. An unknown slug fails fast and lists the valid options. Ignored when 'Sweep all categories' is enabled.

## `allCategories` (type: `boolean`):

Iterate every trending category for the chosen country (111 categories x 20 rows). Combine with 'Max results' to cap the run.

## `includeHistory` (type: `boolean`):

Include the monthly history arrays for search traffic, visits, device split, backlinks and referring domains. Turn off for smaller, flatter rows.

## `maxItems` (type: `integer`):

Stop after this many rows have been pushed to the dataset.

## `minRequestInterval` (type: `number`):

Politeness delay before each domain lookup. Lower is faster; 0.5 is a good default.

## `proxyConfiguration` (type: `object`):

Optional. Semrush's public pages do not require a proxy, but you can route through Apify Proxy if you prefer.

## Actor input object example

```json
{
  "mode": "domains",
  "domains": [
    "stripe.com",
    "shopify.com",
    "nytimes.com"
  ],
  "country": "us",
  "category": "finance",
  "allCategories": false,
  "includeHistory": true,
  "maxItems": 100,
  "minRequestInterval": 0.5,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

One flat row per domain — country and category rank, organic and paid search traffic, visits and desktop/mobile split, backlinks and referring domains with month-over-month change, plus 6 months of history.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "stripe.com",
        "shopify.com",
        "nytimes.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("axlymxp/semrush-website-traffic-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "stripe.com",
        "shopify.com",
        "nytimes.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("axlymxp/semrush-website-traffic-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "stripe.com",
    "shopify.com",
    "nytimes.com"
  ]
}' |
apify call axlymxp/semrush-website-traffic-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,axlymxp/semrush-website-traffic-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RCXn6cUI6Sg6fBZMD/builds/n2FGWoY9pVtQZTbLX/openapi.json