Semrush Authority Score Scraper — Backlinks & Traffic

Overview

Semrush Authority Score Scraper turns any domain into a clean, structured JSON record pulled from Semrush's free Website Authority Checker tool. Every row includes the Authority Score (1–100), total backlinks, referring domains, an estimated monthly organic traffic figure, twelve months of historical trend data, the domain's top pages by backlinks, and its top five organic competitors with their own Authority Scores.

No Semrush account. No login. No paid Semrush Data API subscription. This actor reads exactly what Semrush's own public, unauthenticated marketing tool shows any visitor for free, and turns it into a dataset you can export, join, or feed into an agent.

✅ No login required | ✅ No API key | ✅ No Semrush subscription | ✅ MCP-ready for AI agents

Coverage — please read before buying

This is important, so we are stating it plainly: this actor scrapes Semrush's free preview tool, not the paid Semrush platform. It does not and cannot return anything from Semrush's paid dashboard, Data API, or Pro/Guru/Business plans — full keyword lists, complete backlink exports, position tracking, keyword gap analysis, and traffic-by-country breakdowns all live behind a Semrush subscription and are genuinely out of scope here. If you need the full Semrush suite, you need a Semrush account.

What you get instead is exactly what semrush.com/free-tools/website-authority-checker/ shows an anonymous visitor, structured as JSON:

• Authority Score (1–100) and its recent trend
• Total backlinks and their recent trend
• Total referring domains and their recent trend
• An estimated monthly organic traffic figure
• Up to 13 months of historical Authority Score, backlink, and referring-domain data points
• Up to 5 pages on the domain with the most raw backlinks
• Up to 5 pages on the domain with the most unique referring domains
• Up to 5 competing domains, each with its own Authority Score and a shared-keyword count
• A link-power / natural-profile / organic-traffic score breakdown with a qualitative health label

There is no separate "organic keyword count" field for the target domain itself — Semrush's free tool does not expose one. If your workflow needs the actual keyword list, keyword volumes, or SERP positions, this is the wrong tool; those require a Semrush subscription.

Known reliability risk — read this before running

Semrush protects this endpoint with Google reCAPTCHA v3, which scores every request 0–1 on a continuous, mostly IP-reputation-driven risk model rather than a one-time puzzle. In live testing during this actor's build, requests routed through Apify's proxy infrastructure — both RESIDENTIAL sessions and Apify's own direct egress — consistently scored in the 0.0–0.3 "suspicious" range and were rejected, across five independently-configured attempts (different stealth/fingerprint settings, memory tiers, proxy types, and human-paced interaction timing). The same flow succeeds reliably from a browser on a clean, non-shared-proxy network. We are stating this plainly rather than claiming reliability we have not observed: treat this as an actor whose success rate depends on Apify's proxy IP reputation at the moment you run it, not a guaranteed-to-work scraper. Blocked and rejected checks are never charged, so a low-scoring run costs you nothing — but budget for the possibility that a given run delivers 0 of N domains.

How it works

Semrush's free Website Authority Checker is served from a public, no-login marketing page, but the JSON endpoint behind it is not casually scriptable. Plain HTTP requests (curl, a bare fetch, got, etc.) are rejected outright with 403 Access denied, even with realistic headers and a warm session cookie — so a real browser is required, full stop. But a real browser alone is not sufficient either: the endpoint also requires a live Google reCAPTCHA v3 token, and Google scores every request 0–1 for risk before Semrush will return data. That score leans heavily on network-level signals (IP reputation) in addition to in-browser behavior.

So this actor drives a real, headed Chromium browser (via Crawlee's PlaywrightCrawler) over an Apify proxy. For each domain, it opens the tool page in a fresh browser session, fills the domain into the real input field, clicks the real "Check Authority" button with human-paced timing, and reads the result back through Playwright's own network listener — the exact interaction a person would perform, not a re-implemented API call.

The free tool is also rate-limited to 3 checks per day per source IP. A fresh cookie jar from the same IP does not reset it; only a new IP does. That is why this actor requests a brand-new proxy session (and therefore a fresh exit IP) for every single domain, rather than reusing one session across a batch — so a run checking 20 domains isn't racing to exhaust one IP's daily allowance, it spends each domain against its own untouched one.

When a request scores well, you get the same public data a human would see by visiting the tool and clicking "Check Authority." When it doesn't, see "Known reliability risk" above — you are not charged for it.

🧾 Input configuration

{
  "domains": ["hubspot.com", "salesforce.com"],
  "maxDomains": 20,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Field	Description
domains	Array of domains to check (protocol and www. are stripped automatically).
maxDomains	Safety cap on how many domains from the list are actually checked in one run. Default 20.
proxyConfiguration	RESIDENTIAL proxy is required — the target blocks non-browser traffic and rate-limits per IP.

📤 Output format

{
  "domain": "hubspot.com",
  "data_as_of": "2026-08-01",
  "authority_score": 79,
  "authority_score_change_pct": 0,
  "backlinks": 79228382,
  "backlinks_change_pct": -7.24,
  "referring_domains": 421085,
  "referring_domains_change_pct": -3.22,
  "organic_traffic_monthly": 248376,
  "organic_traffic_change_pct": 73.58,
  "health_label": "Very good",
  "link_power_score": 6.99,
  "natural_profile_score": 6.0,
  "organic_traffic_score": 5.68,
  "top_pages_by_backlinks": [
    { "url": "https://meetings.hubspot.com/andrea873", "referring_domains": 3, "backlinks": 1660245 }
  ],
  "most_linked_pages": [
    { "url": "https://www.hubspot.com/", "referring_domains": 66453, "backlinks": 849049 }
  ],
  "top_competitors": [
    { "domain": "salesforce.com", "authority_score": 84, "common_keywords": 25208 }
  ],
  "authority_score_history": [
    { "date": "2026-08-01", "value": 79 },
    { "date": "2026-07-01", "value": 79 }
  ],
  "checked_at": "2026-08-02T14:22:54.512Z"
}

Every field is described in the dataset schema tab. Fields the tool does not return for a given domain are omitted rather than sent as empty strings, so downstream parsing stays clean.

💼 Common use cases

Quick competitive snapshots Pull Authority Score, backlinks, and traffic estimates for a shortlist of competitors before deciding which one deserves a deeper (paid) audit.

Prospect and vendor screening Before a guest-post pitch, link exchange, partnership, or vendor evaluation, get an instant read on a domain's authority and backlink health.

Lightweight SEO monitoring Track your own domain's Authority Score and backlink trend over time without paying for a full SEO suite seat.

Enrichment for lead and account lists Attach a domain-authority signal to a list of company websites as a cheap qualification layer.

FAQ

Do I need a Semrush account or API key? No. This actor reads Semrush's public, unauthenticated Website Authority Checker — the same tool anyone can use at semrush.com/free-tools/website-authority-checker/ without signing up.

Is this the same as Semrush's full Domain Overview or Data API? No, and we want to be direct about that. Semrush's main analytics dashboard (semrush.com/analytics/overview/) requires an account for any unauthenticated visitor — it shows a "Register to get 10 free requests" wall with no data behind it. This actor does not touch that surface or Semrush's paid Data API. It only returns what the free Website Authority Checker exposes.

Why does it need a residential proxy? The free tool's API blocks plain HTTP outright and rate-limits to 3 checks per day per source IP. A residential proxy with a fresh session per domain is what makes checking more than 3 domains in one run possible at all.

How is pricing calculated? Pay per event. You are charged only for each domain successfully delivered. Blocked, rate-limited, or failed checks are never charged. There is no free tier.

How often does the underlying data update? Semrush states it refreshes Authority Scores for all domains roughly every two weeks, pulled directly from its own backlink and traffic databases.

Use in Claude, ChatGPT & any MCP agent

https://mcp.apify.com/?tools=themineworks/semrush-scraper

Or call it from code with the Apify client:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/semrush-scraper').call({
  domains: ['hubspot.com', 'salesforce.com'],
  maxDomains: 20,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

🛠️ Complete your SEO intelligence pipeline

Pair this scraper with the rest of the themineworks competitive-intelligence suite for a fuller picture beyond one free-tool snapshot:

• Crunchbase Scraper: company funding, investors, and firmographics.
• AmbitionBox Scraper: company ratings, reviews, and salary data.

Typical flow: use this actor for a fast, free authority/traffic read on a shortlist of domains, then move only the domains worth deeper research into a full paid SEO tool.

Found a bug or have a feature request? Open an issue on the actor's Apify Console page or reach out through the Apify profile.