# Semrush Authority Score Scraper — Backlinks & Traffic (`themineworks/semrush-scraper`) Actor

Get Semrush Authority Score, backlinks, referring domains, and organic traffic estimates for any domain via Semrush's free Website Authority Checker. No Semrush account, login, or paid API needed. Use it as an MCP server in Claude, ChatGPT & AI agents.

- **URL**: https://apify.com/themineworks/semrush-scraper.md
- **Developed by:** [The Mine Works](https://apify.com/themineworks) (community)
- **Categories:** SEO tools, MCP servers, Developer tools
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $9.00 / 1,000 domain checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Semrush Authority Score Scraper — Backlinks & Traffic

### Overview

Semrush Authority Score Scraper turns any domain into a clean, structured JSON record pulled from Semrush's free **Website Authority Checker** tool. Every row includes the Authority Score (1–100), total backlinks, referring domains, an estimated monthly organic traffic figure, twelve months of historical trend data, the domain's top pages by backlinks, and its top five organic competitors with their own Authority Scores.

No Semrush account. No login. No paid Semrush Data API subscription. This actor reads exactly what Semrush's own public, unauthenticated marketing tool shows any visitor for free, and turns it into a dataset you can export, join, or feed into an agent.

✅ No login required | ✅ No API key | ✅ No Semrush subscription | ✅ MCP-ready for AI agents

### Coverage — please read before buying

This is important, so we are stating it plainly: **this actor scrapes Semrush's free preview tool, not the paid Semrush platform.** It does not and cannot return anything from Semrush's paid dashboard, Data API, or Pro/Guru/Business plans — full keyword lists, complete backlink exports, position tracking, keyword gap analysis, and traffic-by-country breakdowns all live behind a Semrush subscription and are genuinely out of scope here. If you need the full Semrush suite, you need a Semrush account.

What you get instead is exactly what `semrush.com/free-tools/website-authority-checker/` shows an anonymous visitor, structured as JSON:

- Authority Score (1–100) and its recent trend
- Total backlinks and their recent trend
- Total referring domains and their recent trend
- An estimated monthly organic traffic figure
- Up to 13 months of historical Authority Score, backlink, and referring-domain data points
- Up to 5 pages on the domain with the most raw backlinks
- Up to 5 pages on the domain with the most unique referring domains
- Up to 5 competing domains, each with its own Authority Score and a shared-keyword count
- A link-power / natural-profile / organic-traffic score breakdown with a qualitative health label

There is no separate "organic keyword count" field for the target domain itself — Semrush's free tool does not expose one. If your workflow needs the actual keyword list, keyword volumes, or SERP positions, this is the wrong tool; those require a Semrush subscription.

#### Known reliability risk — read this before running

Semrush protects this endpoint with Google reCAPTCHA v3, which scores every request 0–1 on a continuous, mostly IP-reputation-driven risk model rather than a one-time puzzle. In live testing during this actor's build, requests routed through Apify's proxy infrastructure — both RESIDENTIAL sessions and Apify's own direct egress — consistently scored in the 0.0–0.3 "suspicious" range and were rejected, across five independently-configured attempts (different stealth/fingerprint settings, memory tiers, proxy types, and human-paced interaction timing). The same flow succeeds reliably from a browser on a clean, non-shared-proxy network. We are stating this plainly rather than claiming reliability we have not observed: **treat this as an actor whose success rate depends on Apify's proxy IP reputation at the moment you run it, not a guaranteed-to-work scraper.** Blocked and rejected checks are never charged, so a low-scoring run costs you nothing — but budget for the possibility that a given run delivers 0 of N domains.

### How it works

Semrush's free Website Authority Checker is served from a public, no-login marketing page, but the JSON endpoint behind it is not casually scriptable. Plain HTTP requests (curl, a bare `fetch`, `got`, etc.) are rejected outright with `403 Access denied`, even with realistic headers and a warm session cookie — so a real browser is required, full stop. But a real browser alone is not sufficient either: the endpoint also requires a live Google reCAPTCHA v3 token, and Google scores every request 0–1 for risk before Semrush will return data. That score leans heavily on network-level signals (IP reputation) in addition to in-browser behavior.

So this actor drives a real, headed Chromium browser (via Crawlee's `PlaywrightCrawler`) over an Apify proxy. For each domain, it opens the tool page in a fresh browser session, fills the domain into the real input field, clicks the real "Check Authority" button with human-paced timing, and reads the result back through Playwright's own network listener — the exact interaction a person would perform, not a re-implemented API call.

The free tool is also rate-limited to **3 checks per day per source IP**. A fresh cookie jar from the same IP does not reset it; only a new IP does. That is why this actor requests a brand-new proxy session (and therefore a fresh exit IP) for every single domain, rather than reusing one session across a batch — so a run checking 20 domains isn't racing to exhaust one IP's daily allowance, it spends each domain against its own untouched one.

When a request scores well, you get the same public data a human would see by visiting the tool and clicking "Check Authority." When it doesn't, see "Known reliability risk" above — you are not charged for it.

### 🧾 Input configuration

```json
{
  "domains": ["hubspot.com", "salesforce.com"],
  "maxDomains": 20,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}
```

| Field | Description |
| --- | --- |
| `domains` | Array of domains to check (protocol and `www.` are stripped automatically). |
| `maxDomains` | Safety cap on how many domains from the list are actually checked in one run. Default 20. |
| `proxyConfiguration` | RESIDENTIAL proxy is required — the target blocks non-browser traffic and rate-limits per IP. |

### 📤 Output format

```json
{
  "domain": "hubspot.com",
  "data_as_of": "2026-08-01",
  "authority_score": 79,
  "authority_score_change_pct": 0,
  "backlinks": 79228382,
  "backlinks_change_pct": -7.24,
  "referring_domains": 421085,
  "referring_domains_change_pct": -3.22,
  "organic_traffic_monthly": 248376,
  "organic_traffic_change_pct": 73.58,
  "health_label": "Very good",
  "link_power_score": 6.99,
  "natural_profile_score": 6.0,
  "organic_traffic_score": 5.68,
  "top_pages_by_backlinks": [
    { "url": "https://meetings.hubspot.com/andrea873", "referring_domains": 3, "backlinks": 1660245 }
  ],
  "most_linked_pages": [
    { "url": "https://www.hubspot.com/", "referring_domains": 66453, "backlinks": 849049 }
  ],
  "top_competitors": [
    { "domain": "salesforce.com", "authority_score": 84, "common_keywords": 25208 }
  ],
  "authority_score_history": [
    { "date": "2026-08-01", "value": 79 },
    { "date": "2026-07-01", "value": 79 }
  ],
  "checked_at": "2026-08-02T14:22:54.512Z"
}
```

Every field is described in the dataset schema tab. Fields the tool does not return for a given domain are omitted rather than sent as empty strings, so downstream parsing stays clean.

### 💼 Common use cases

**Quick competitive snapshots**
Pull Authority Score, backlinks, and traffic estimates for a shortlist of competitors before deciding which one deserves a deeper (paid) audit.

**Prospect and vendor screening**
Before a guest-post pitch, link exchange, partnership, or vendor evaluation, get an instant read on a domain's authority and backlink health.

**Lightweight SEO monitoring**
Track your own domain's Authority Score and backlink trend over time without paying for a full SEO suite seat.

**Enrichment for lead and account lists**
Attach a domain-authority signal to a list of company websites as a cheap qualification layer.

### FAQ

**Do I need a Semrush account or API key?**
No. This actor reads Semrush's public, unauthenticated Website Authority Checker — the same tool anyone can use at semrush.com/free-tools/website-authority-checker/ without signing up.

**Is this the same as Semrush's full Domain Overview or Data API?**
No, and we want to be direct about that. Semrush's main analytics dashboard (`semrush.com/analytics/overview/`) requires an account for any unauthenticated visitor — it shows a "Register to get 10 free requests" wall with no data behind it. This actor does not touch that surface or Semrush's paid Data API. It only returns what the free Website Authority Checker exposes.

**Why does it need a residential proxy?**
The free tool's API blocks plain HTTP outright and rate-limits to 3 checks per day per source IP. A residential proxy with a fresh session per domain is what makes checking more than 3 domains in one run possible at all.

**How is pricing calculated?**
Pay per event. You are charged only for each domain successfully delivered. Blocked, rate-limited, or failed checks are never charged. There is no free tier.

**How often does the underlying data update?**
Semrush states it refreshes Authority Scores for all domains roughly every two weeks, pulled directly from its own backlink and traffic databases.

### Use in Claude, ChatGPT & any MCP agent

```
https://mcp.apify.com/?tools=themineworks/semrush-scraper
```

Or call it from code with the Apify client:

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('themineworks/semrush-scraper').call({
  domains: ['hubspot.com', 'salesforce.com'],
  maxDomains: 20,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);
```

### 🛠️ Complete your SEO intelligence pipeline

Pair this scraper with the rest of the themineworks competitive-intelligence suite for a fuller picture beyond one free-tool snapshot:

- **[Crunchbase Scraper](https://apify.com/themineworks/crunchbase-companies)**: company funding, investors, and firmographics.
- **[AmbitionBox Scraper](https://apify.com/themineworks/ambitionbox-companies)**: company ratings, reviews, and salary data.

Typical flow: use this actor for a fast, free authority/traffic read on a shortlist of domains, then move only the domains worth deeper research into a full paid SEO tool.

Found a bug or have a feature request? Open an issue on the actor's Apify Console page or reach out through the Apify profile.

# Actor input Schema

## `domains` (type: `array`):

Website domains to look up on Semrush's free Website Authority Checker (e.g. "hubspot.com"). Protocol and "www." are stripped automatically. One dataset row is returned per domain.

## `maxDomains` (type: `integer`):

Safety cap on how many domains from the list above are actually checked in one run. Each domain needs its own fresh proxy IP and takes roughly 10–20 seconds, so keep this small for quick runs.

## `proxyConfiguration` (type: `object`):

Semrush's free-tools API rate-limits to 3 checks per day PER SOURCE IP and blocks non-browser HTTP traffic outright. RESIDENTIAL proxy is required — the actor requests a fresh session (fresh exit IP) for every domain so each check gets its own untouched daily allowance.

## Actor input object example

```json
{
  "domains": [
    "hubspot.com"
  ],
  "maxDomains": 1,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "hubspot.com"
    ],
    "maxDomains": 1,
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": [
            "RESIDENTIAL"
        ]
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("themineworks/semrush-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": ["hubspot.com"],
    "maxDomains": 1,
    "proxyConfiguration": {
        "useApifyProxy": True,
        "apifyProxyGroups": ["RESIDENTIAL"],
    },
}

# Run the Actor and wait for it to finish
run = client.actor("themineworks/semrush-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "hubspot.com"
  ],
  "maxDomains": 1,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "RESIDENTIAL"
    ]
  }
}' |
apify call themineworks/semrush-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,themineworks/semrush-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/OiFz3aTWiTgpbtliV/builds/uCsUKELHrPaInY3BA/openapi.json