# 🧪 Semrush Competitors Scraper — Rivals Side by Side (`thenetaji/semrush-competitors-scraper`) Actor

Export the rivals Semrush names for any website, each carrying the website's own metric set — visits, authority score, bounce rate, pages per visit, time on site and all three ranks — so they line up in one table. The website itself is saved as a row beside them.

- **URL**: https://apify.com/thenetaji/semrush-competitors-scraper.md
- **Developed by:** [The Netaji](https://apify.com/thenetaji) (community)
- **Categories:** SEO tools, Marketing, Business
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.28 / 1,000 comparable profiles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Semrush Competitors Scraper

The rivals Semrush names for a website, each carrying the website's own metric set, so they
line up in one table you can sort. Every row — the website you asked about and each of its
rivals — has visits and their month-on-month movement, the authority score, bounce rate,
pages per visit, time on site, and the global, country and industry rank.

The website itself is saved as a row beside its rivals, marked `subject`. A dataset of
competitors with nothing to compare them against is half an answer.

### Accepted input

`domains` is required and takes one website per line, either as a domain such as
`stripe.com` or as any link to the site. A link is reduced to its host, a `www.` prefix is
stripped because Semrush publishes a page for `stripe.com` and none for `www.stripe.com`,
and any other subdomain has no page of its own and is skipped with a line in the run log.

`maxItems` defaults to `100` and caps **rows across the whole run**, not websites. One
website produces its own row plus one per rival — between three and seven, measured — so a
list of ten websites is roughly forty to eighty rows. `0` means no limit.

```json
{
  "domains": ["stripe.com", "shopify.com"],
  "maxItems": 100
}
```

### Response fields

```json
{
  "requested_domain": "stripe.com",
  "url": "https://www.semrush.com/website/stripe.com/competitors/",
  "updated_at": "2026-08-12",
  "domain": "gocardless.com",
  "relation": "competitor",
  "as_of": "2026-07-15",
  "database": "US",
  "visits": 3184220,
  "visits_change_percent": -0.041,
  "authority_score": 78,
  "bounce_rate": 0.5413,
  "pages_per_visit": 3.9,
  "time_on_site_seconds": 214,
  "global_rank": 18422,
  "country_rank": 9930,
  "country_code": "US",
  "country_name": "United States",
  "category_rank": 61,
  "category_name": "Finance",
  "category_slug": "finance",
  "competitor_count": 3
}
```

Fields Semrush states no figure for are returned as null rather than omitted, so every row
has the same shape — which is the whole point here, because rows of different shapes cannot
be compared.

`relation` is `subject` on the website you asked about and `competitor` on each rival. It is
the only field that separates them; everything else is deliberately identical.

`requested_domain` is on every row, so a run over twenty websites splits back apart into
twenty comparison tables without any guessing about which rival belonged to which subject.

`competitor_count` is counted from what actually arrived, never copied from a figure Semrush
stated. How many rivals a website has is not a parameter and cannot be raised — it measured
between three and seven depending on the website.

`bounce_rate` is a **fraction**: `0.5413` is 54.13%. So is `visits_change_percent`. The three
ranks are positions, where a smaller number is better.

### Behaviour on partial results

A website Semrush has no page for is skipped with a line in the run log and the rest of the
list still runs. An entry that is not a domain at all is skipped before any request is made.

A website Semrush names **no** rivals for still produces its own `subject` row, with
`competitor_count` at `0`. That is deliberate: the subject row is a figure to compare against
later, and an empty response is not.

Individual metrics can be null on a rival Semrush states no figure for. `null` is never
rendered as zero — a zero visits figure would be a measurement nobody made — so a rival with
a null `visits` is a rival Semrush declines to estimate, not a rival with no traffic.

### Frequently asked questions

**How is this different from the competitors panel on the website profile?**
That panel gives a name, a similarity score and a visits figure that is frequently null. This
gives every rival the subject's full metric set — visits, authority score, bounce rate, pages
per visit, time on site and all three ranks — which is what makes a comparison table
possible. If all you need is a list of names, the panel on
[Semrush Website Scraper](https://apify.com/thenetaji/semrush-website-scraper) already has
it and costs you nothing extra.

**Can I ask for more competitors?**
No. How many Semrush names is not a parameter and is not stated by the page; it is what
arrived, and it measured between three and seven. An input for it would be a control that
does nothing.

**Are these live numbers?**
No. They are monthly estimates, and every row carries the month it describes in `as_of`. Two
runs a week apart return the same figures.

**Why is there no country selector?**
Because the free page has one database and it does not change. The same page fetched from US,
GB, DE and IN exits came back identical within two bytes, and Semrush's own database
parameters were ignored. Each row states what its figures are measured in, under `database`.

**How do I turn this into a comparison table?**
Group the dataset by `requested_domain`. Each group is one subject row and its rivals, in the
same columns, already sorted by whatever your spreadsheet sorts by. `relation` marks the
subject.

**Is a Semrush account or API key required?**
No. No account, session cookie or key of any kind is supplied to the Actor or needed by it.
Open the `url` on any row and you see the same numbers with no login.

### Related Actors

[Semrush Website Scraper](https://apify.com/thenetaji/semrush-website-scraper) reads one
website in full rather than comparably: top keywords with volume and cost per click,
backlinks and referring domains with their histories, the country split, the traffic journey
and referral traffic from each AI assistant. Every `domain` on a row here is a valid input
for it, so a competitor set is a starting point rather than a dead end.

[Semrush Top Websites](https://apify.com/thenetaji/semrush-trending-scraper) returns the
hundred most visited websites for a country and an industry, which is where a competitor list
comes from when you do not already have one.

# Actor input Schema

## `domains` (type: `array`):

The websites to look up, one per line, as a domain such as stripe.com or as any link to the site. A full link is accepted and reduced to its host. A `www.` prefix is removed, because the site publishes a page for `stripe.com` and none for `www.stripe.com`; any other subdomain genuinely has no page of its own and comes back empty rather than being quietly rewritten to its parent. These are monthly estimates, not live counters — every row carries the month it describes in `as_of`, and two runs a week apart return the same figures.

## `maxItems` (type: `integer`):

Maximum rows to save across the whole run. Set 0 for no limit. One website produces its own row plus one per rival — measured between 3 and 7 rivals — so a list of ten websites is roughly forty to eighty rows.

## Actor input object example

```json
{
  "domains": [
    "stripe.com"
  ],
  "maxItems": 20
}
```

# Actor output Schema

## `dataset` (type: `string`):

All records scraped by this run

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "stripe.com",
        "gocardless.com"
    ],
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("thenetaji/semrush-competitors-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": [
        "stripe.com",
        "gocardless.com",
    ],
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("thenetaji/semrush-competitors-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "stripe.com",
    "gocardless.com"
  ],
  "maxItems": 20
}' |
apify call thenetaji/semrush-competitors-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,thenetaji/semrush-competitors-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/NfX0aduSamyIgkUwL/builds/5p7GhiHAhGwjGkiUU/openapi.json