Semrush Website Traffic Scraper

Pull Semrush website analytics in bulk — country and category rank, organic and paid search traffic, desktop/mobile split, and 6 months of backlink and referring-domain history — for any list of domains. No Semrush account, no login, no CAPTCHA, no proxy required.

Point it at a list of domains, or let it discover trending websites from Semrush's own rankings across 111 industry categories and export the winners automatically.

Who it's for

• SEO agencies & consultants — qualify inbound leads and build competitive pitch decks with real rank, traffic and backlink numbers instead of screenshots.
• Growth & marketing teams — track the traffic trajectory of your competitors month over month.
• Investors & analysts — size a market or diligence a company from traffic trend data.
• Lead generation — score thousands of prospect domains by traffic and authority, then filter to the ones worth contacting.
• Data teams & researchers — build a longitudinal traffic dataset by scheduling monthly runs.

What makes it different

Most Semrush actors need a paid Semrush login (and often a pool of accounts plus residential proxies) to work — which is exactly why they break. This actor reads only Semrush's public, server-rendered website overview pages, so there is nothing to log into and nothing to get banned. That includes the full backlink and referring-domain history, which the public pages expose as monthly series.

Metrics that Semrush genuinely keeps behind its paywall (keyword lists, competitor sets, site audit, traffic-by-country, traffic journey) are returned as null rather than guessed or faked.

Output fields

Each row is one domain, flattened and ready for a spreadsheet or database.

Identity & categorization

Field	Type	Description
domain	string	Root domain analyzed, e.g. stripe.com.
url	string	Semrush overview page the data came from.
brand_name	string | null	Brand name Semrush associates with the domain.
country	string | null	ISO code of the domain's primary country.
categories	array	Semrush industry category names.
category_slugs	array	URL slugs of the category names.
favicon	string | null	Favicon image URL.
data_date	string | null	Date the metrics refer to (Semrush lags ~1 month).
date_modified	string | null	Date Semrush last refreshed the page.

Rankings

Field	Type	Description
global_rank	integer | null	Global traffic rank (often null on the public page).
global_rank_trend	string | null	up, flat or down.
country_rank	integer | null	Traffic rank within the domain's country.
country_rank_country	string | null	Country the rank applies to.
country_rank_trend	string | null	up, flat or down.
category_rank	integer | null	Rank within the primary industry category.
category_rank_category	string | null	Category the rank applies to.
category_rank_trend	string | null	up, flat or down.

Search traffic

Field	Type	Description
search_traffic_date	string | null	Date of the search-traffic figures.
organic_traffic	number | null	Estimated monthly organic search visits.
organic_traffic_change_percent	number | null	Month-over-month change in organic traffic (%).
paid_traffic	number | null	Estimated monthly paid search visits.
paid_traffic_change_percent	number | null	Month-over-month change in paid traffic (%).
total_search_traffic	number | null	Organic plus paid search visits.

Visits & device split

Field	Type	Description
traffic_date	string | null	Date of the visits/device figures.
visits	number | null	Estimated total monthly visits.
visits_change_percent	number | null	Month-over-month change in visits (%).
desktop_visits	number | null	Monthly visits from desktop.
mobile_visits	number | null	Monthly visits from mobile.
desktop_share_percent	number | null	Desktop share of visits (%).
mobile_share_percent	number | null	Mobile share of visits (%).

Backlinks & authority

Field	Type	Description
backlinks	number | null	Total backlinks in the most recent month.
backlinks_previous_month	number | null	Total backlinks in the prior month.
backlinks_change_percent	number | null	Month-over-month change in backlinks (%).
referring_domains	number | null	Unique referring domains in the most recent month.
referring_domains_previous_month	number | null	Unique referring domains in the prior month.
referring_domains_change_percent	number | null	Month-over-month change in referring domains (%).
google_penalty_risk	string | null	Semrush's Google penalty risk label, e.g. Low.

History arrays (set includeHistory to true)

Field	Type	Description
search_traffic_history	array	Monthly organic/paid/total search traffic, most recent first.
traffic_by_device_history	array	Monthly visits split into desktop and mobile, most recent first.
visits_history	array	Monthly total visits, most recent first.
backlinks_history	array	Monthly backlink counts, most recent first.
referring_domains_history	array	Monthly referring-domain counts, most recent first.

Trending context (trending mode only)

Field	Type	Description
trending_rank	integer | null	Position in the Semrush trending ranking.
trending_country	string | null	Country of the trending ranking.
trending_category	string | null	Category slug of the trending ranking.
trending_total_traffic	string | null	Traffic as displayed in the trending table, e.g. 24.54B.
trending_desktop_share_percent	number | null	Desktop share shown in the trending table.
trending_mobile_share_percent	number | null	Mobile share shown in the trending table.
trending_mom_percent	number | null	Month-over-month traffic change from the trending table.
trending_yoy_percent	number | null	Year-over-year traffic change from the trending table.
trending_top_source	string | null	Largest traffic source, e.g. Direct.

Run metadata

Field	Type	Description
source_mode	string	Discovery mode that produced the row (domains or trending).
found	boolean	false when Semrush had no overview data for the domain.
scraped_at	string	UTC timestamp when the row was scraped (ISO-8601).

Use cases

Rank a lead list by traffic before you call anyone. Feed 5,000 prospect domains in domains mode, sort the dataset by visits and backlinks, and route only the top slice to sales.

Build a competitor traffic tracker. Schedule a monthly run over your competitor set and append to the same dataset — visits_change_percent, organic_traffic_change_percent and backlinks_change_percent give you the deltas without any extra maths.

Find category winners you've never heard of. Turn on allCategories for a country and get the top 20 trending sites in each of 111 categories, enriched with full overview metrics.

Diligence a target company. backlinks_history and visits_history show whether growth is real and sustained or a one-month spike.

Spot link-building momentum. referring_domains_change_percent reveals who is actively acquiring links right now — useful for both competitive intel and outreach targeting.

Input parameters

Parameter	Type	Default	Description
mode	string	domains	domains analyzes the list you provide; trending discovers domains from Semrush's trending rankings.
domains	array	—	Domains or URLs to analyze, e.g. stripe.com or https://www.nytimes.com/section/world. URLs are normalized to the root domain automatically.
country	string	us	Country code for the trending ranking, e.g. us, gb, de, in.
category	string	all	Trending category slug, e.g. finance, travel-and-tourism, mass-media. An unknown slug fails fast and lists the valid options.
allCategories	boolean	false	Sweep every trending category for the chosen country (111 categories × 20 rows).
includeHistory	boolean	true	Include the 6-month history arrays. Turn off for a compact, flat CSV.
maxItems	integer	100	Stop after this many rows.
minRequestInterval	number	0.5	Politeness delay in seconds before each domain lookup.
proxyConfiguration	object	disabled	Optional. Not required — the public pages are not proxy-gated.

Example input — domain list

{
    "mode": "domains",
    "domains": ["stripe.com", "shopify.com", "https://www.nytimes.com/section/world"],
    "includeHistory": true,
    "maxItems": 100
}

Example input — trending discovery

{
    "mode": "trending",
    "country": "us",
    "category": "finance",
    "includeHistory": false,
    "maxItems": 20
}

Example output row (truncated)

{
    "domain": "stripe.com",
    "url": "https://www.semrush.com/website/stripe.com/overview/",
    "country": "US",
    "categories": ["Computer Software and Development", "Finance"],
    "data_date": "2026-08-15",
    "country_rank": 161,
    "country_rank_country": "United States",
    "country_rank_trend": "down",
    "category_rank": 6,
    "category_rank_category": "Finance",
    "organic_traffic": 5667468,
    "organic_traffic_change_percent": 2.48,
    "paid_traffic": 439916,
    "visits": 203227314,
    "visits_change_percent": 12.94,
    "desktop_share_percent": 51.25,
    "mobile_share_percent": 48.75,
    "backlinks": 151822916,
    "backlinks_previous_month": 167449561,
    "backlinks_change_percent": -9.33,
    "referring_domains": 664614,
    "referring_domains_change_percent": 4.86,
    "google_penalty_risk": "Low",
    "backlinks_history": [
        { "date": "2026-07-01", "backlinks": 151822916 },
        { "date": "2026-06-01", "backlinks": 167449561 }
    ],
    "found": true,
    "scraped_at": "2026-09-17T02:12:00Z"
}

Scheduling & webhooks

Semrush refreshes these pages roughly monthly, so a monthly schedule is the natural cadence for trend tracking. In the Apify Console open Schedules → Create new, pick this actor, and save your input.

Add a webhook (Actor → Integrations) on ACTOR.RUN.SUCCEEDED to push each finished dataset straight into Google Sheets, Slack, Zapier, Make, or your own endpoint. Runs resume from a checkpoint if they are aborted and resurrected, so long sweeps pick up where they stopped instead of starting over.

Using it from an AI assistant (MCP)

This actor works as a tool for LLM agents through the Apify MCP server, so an assistant can pull live traffic data mid-conversation:

{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": ["-y", "@apify/actors-mcp-server"],
            "env": { "APIFY_TOKEN": "<YOUR_APIFY_TOKEN>" }
        }
    }
}

Then ask things like "Compare the organic traffic and backlink growth of stripe.com and adyen.com over the last six months."

You can also call it from code with the Apify API or the JS/Python clients — every run's output is a standard dataset available as JSON, CSV, Excel or XML.

FAQ

Do I need a Semrush subscription or login? No. The actor only reads Semrush's public website overview and trending pages. There is no account, API key, or cookie to supply.

Do I need proxies? No. The pages are not proxy-gated. A proxyConfiguration input exists if you want to route traffic through Apify Proxy anyway, but leaving it off is fine and cheaper.

Why are authority_score, keywords, or competitor lists missing? Those are genuinely paywalled by Semrush and are not present in the public page data. This actor returns null for what it cannot see rather than inventing values. Backlinks and referring domains are available, including their monthly history.

How fresh is the data? Semrush's public overview data lags real time by roughly one month. Every row carries data_date and date_modified so you always know the vintage.

How accurate is it? The numbers are exactly what Semrush publishes — this actor does no estimation of its own. Semrush's figures are themselves modelled estimates, so treat them as directional, as you would in the Semrush UI.

What happens with a domain Semrush doesn't track? The run continues. In domains mode the domain is skipped and logged; in trending mode the row is still emitted with found: false so you keep the ranking context.

Can it handle thousands of domains? Yes. Raise maxItems, and the run checkpoints its progress so an aborted-and-resurrected run continues rather than restarting.

Is scraping this legal? The actor reads only publicly accessible pages — no login is bypassed and no paywalled content is unlocked. You are responsible for using the output in line with Semrush's terms and the laws that apply to you.

Something broke — how do I get help? Open an issue on the actor's Issues tab with your input JSON and the run ID.