🧪 Semrush Website Scraper — Traffic, Ranks & AI Traffic
Pricing
from $6.80 / 1,000 website profiles
Read any website's whole free Semrush profile as one row: global, country and industry rank, authority score, monthly visits, bounce rate, time on site, top countries, organic and paid search, top keywords with volume and CPC, backlinks, referrers, and referral traffic from each AI assistant.
requested_domain
Optional
The website this row was read for, echoed back after cleaning — so a caller who pasted a link can see what it resolved to.
domain
Optional
The website this row describes. On a competitor row it is the rival rather than the website you asked about; relation says which.
url
Optional
The public page these figures come from. Open it and you see the same numbers, with no account.
as_of
Optional
The month these figures describe. These are monthly estimates, not live counters — every row carries the month it describes in as_of, and two runs a week apart return the same figures.
updated_at
Optional
When the source last refreshed the page these figures were read from.
database
Optional
The country database the country-scoped figures are measured in. It is US on the free page for every website, and the exit country does not change it — which is why there is deliberately no country input on the website and competitor modes that would pretend to.
global_rank
Optional
Position among all websites ranked worldwide.
global_rank_change_places
Optional
How many positions the global rank moved. Stated in places rather than as a percentage, and a NEGATIVE number is an improvement — the rank number fell.
country_rank
Optional
Position within the country named beside it.
country_rank_change_places
Optional
How many positions the country rank moved. Negative is an improvement.
country_code
Optional
The country the country rank is measured in.
country_name
Optional
That country's name.
category_rank
Optional
Position within the industry named beside it.
category_rank_change_places
Optional
How many positions the industry rank moved. Negative is an improvement.
category_name
Optional
The industry the industry rank is measured in.
category_slug
Optional
That industry as a slug, which is the value the trending mode takes.
visits
Optional
Estimated visits in the month named by as_of. Null means the source states no figure for this website, which is not the same as zero — it genuinely has no current estimate for some sites, and unavailable_panels names which sections are affected.
visits_change_percent
Optional
Month-on-month change in visits, as a fraction — 0.1294 is +12.94%.
pages_per_visit
Optional
Average pages viewed per session.
time_on_site_seconds
Optional
Average session duration, in seconds.
bounce_rate
Optional
Bounce rate as a FRACTION, not a percentage: 0.6072 is 60.72%.
organic_search_traffic
Optional
Estimated visits arriving from organic search.
organic_search_change_percent
Optional
Month-on-month change in organic search traffic, as a fraction.
paid_search_traffic
Optional
Estimated visits arriving from paid search.
paid_search_change_percent
Optional
Month-on-month change in paid search traffic, as a fraction.
backlinks
Optional
Total backlinks pointing at the domain.
backlinks_change_percent
Optional
Month-on-month change in backlinks, as a fraction.
referring_domains
Optional
How many distinct domains link to it.
google_penalty_risk
Optional
The stated risk band for the backlink profile.
ai_other_traffic
Optional
Visits from everything other than the AI assistants listed in ai_traffic, for the month. It is the denominator that makes ai_traffic's shares readable.
categories
Optional
Every industry the domain is filed under, each with its name and the slug the trending mode takes.
visits_history
Optional
Up to three months of total visits, one row per month.
device_history
Optional
Up to six months of the desktop-versus-mobile split. Worth reading even when the headline visits is null — it is populated for some domains whose current estimate is not.
top_countries
Optional
The top five countries by traffic, each with its share and its own desktop and mobile split as FRACTIONS. Five is what the free page carries; there is no parameter that returns more.
top_keywords
Optional
The top five organic keywords with position, monthly search volume, cost per click in USD and the share of organic traffic each brings. The intent code is republished as it arrives rather than decoded — no capture has pinned the mapping, and a guessed label is a wrong label that reads like a right one.
keywords_database
Optional
The country database the keyword figures are measured in.
search_traffic_history
Optional
Up to six months of organic and paid search traffic, one row per month.
backlinks_history
Optional
Up to six months of total backlinks.
referring_domains_history
Optional
Up to six months of referring domains.
competitors
Optional
The three rivals the overview page names, each with a similarity score and a visits figure that is frequently null here. The competitors mode returns every rival with the full metric set instead, which is what that mode is for.
traffic_incoming
Optional
The top five sources visitors arrive from. Each carries a domain field that is non-null exactly when that source has a page of its own, so it can be fed straight back into a website run.
traffic_outgoing
Optional
The top five destinations visitors leave for, on the same terms.
ai_traffic
Optional
Referral traffic from individual AI assistants — ChatGPT, Claude, Gemini, Perplexity, Grok and coding agents — each with six months of history. It is the newest thing on the page and it costs nothing to read.
