# Semrush Scraper: Domain Traffic, Authority & Top Websites (`nice_dev/semrush-seo-scraper`) Actor

Semrush domain overview and top websites without an account: Authority Score, visits, ranks, organic and paid traffic, backlinks, top keywords, competitors, AI assistant traffic and 6-month history, plus top sites by country and category. Export to JSON, CSV or Excel.

- **URL**: https://apify.com/nice_dev/semrush-seo-scraper.md
- **Developed by:** [Nice Dev](https://apify.com/nice_dev) (community)
- **Categories:** SEO tools, Marketing, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.07 / 1,000 domain overviews

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### 📈 What is Semrush Scraper?

**Semrush Scraper** extracts the **domain overview of any website from [Semrush](https://www.semrush.com)**: **Authority Score, monthly visits, global and country rank, organic and paid search traffic, backlinks and referring domains, top keywords, competitors, traffic by country and device, traffic sources, and visits sent by AI assistants** (ChatGPT, Gemini, Perplexity…), with 6 months of history. It also lists the **top websites of any country and industry**, as ranked by Semrush. It works as a **Semrush API alternative**: no Semrush subscription and no API units, run it on a schedule, call it from your code, or plug it into Make, Zapier or n8n.

Type a list of **domains** (`stripe.com`, `notion.so`), or pick a **country** and an **industry**, click **Start**, and download the data in JSON, CSV or Excel. No login, nothing to set up. 

### 📋 What data can you extract from Semrush?

One row per domain or per ranked website, 65 fields:

| Category | What you get |
| --- | --- |
| 🌐 **Domain** | domain, brand, main country, industry categories, month of the data |
| 🏆 **Authority and rank** | Semrush Authority Score, global rank, country rank, industry rank — `79`, `#145 worldwide` |
| 👥 **Traffic** | monthly visits and their change, pages per visit, visit duration, bounce rate, desktop vs mobile |
| 🔍 **Search traffic** | organic and paid traffic with their change, the 5 keywords that bring the most visits (position, volume, CPC) |
| 🔗 **Backlinks** | backlinks, referring domains, their change, Google penalty risk |
| 🤖 **AI assistant traffic** | visits sent by ChatGPT, Gemini, Perplexity, Grok… per assistant, with history |
| 🧭 **Traffic sources** | where visitors come from (direct, Google, social…) and where they go next |
| 🌍 **Countries** | the 5 countries sending the most visits, desktop and mobile share for each |
| 🥊 **Competitors** | the 3 closest competitors by shared audience — and, as an option, their own authority, visits and ranks |
| 📅 **History** | 6 months of visits, search traffic, devices, backlinks and referring domains |
| 📊 **Top websites** | rank in a country × industry ranking, visits, month-over-month and year-over-year change, top traffic source |

Every field, with an example, is listed in the **Output** section below.

### ✅ Why use Semrush Scraper?

- 🧾 **The widest domain row**: authority, traffic, search, backlinks, keywords, competitors, countries, sources and AI traffic in one row, with 6-month history — flat columns, ready for a spreadsheet.
- 🤖 **AI assistant traffic**: how many visits ChatGPT, Gemini, Perplexity or Grok send to a site, month by month.
- 📊 **Top websites of 239 countries + worldwide × 110 industries**: the Semrush ranking, about 100 websites per list, with the full overview of each one as an option.
- 🎯 **Paste anything**: a domain, `www.` or a full URL — the Actor reads the root domain and keeps what you typed.
- 💸 **Pay only for data**: a domain Semrush has no data for comes back as a free row that says so.
- 🔔 **Monitoring built in**: tick **Only new rows**, schedule the Actor, and each run returns (and charges) only the rows it has never delivered before.
- 🔌 API, scheduling, monitoring, integrations (Make, Zapier, n8n, Google Sheets…) and JSON/CSV/Excel export via the Apify platform.

### 🚀 How to scrape Semrush

1. Create a free Apify account.
2. Open **Semrush Scraper** and type your **Domains** (e.g. `stripe.com`), one per line.
3. Or pick a **Top websites: country** and one or more **industries** to get the Semrush ranking — or paste Semrush pages into **Semrush URLs**.
4. Set **Max rows** (1000 by default, 0 = no limit), tick the options you want, then click **Start**.
5. Download the dataset in JSON, CSV, Excel or via API.

### 💰 How much does it cost to scrape Semrush?

This Actor uses **pay per event** pricing: you pay per **domain overview** delivered, per **top website** row, and per **competitor detailed** when that option is on, plus a small fee per run start. A domain Semrush has no data for is delivered **free**. Higher Apify plans pay less per row. Platform usage (compute, proxy) is included in the price.

### ⚙️ Input

```json
{
    "domains": ["stripe.com", "notion.so", "leboncoin.fr"],
    "maxItems": 100
}
```

The top websites of two countries in two industries, each with its full overview and its competitors:

```json
{
    "location": "us",
    "locations": ["fr"],
    "topWebsitesCategories": ["finance", "banking"],
    "maxItemsPerQuery": 20,
    "enrichTopWebsites": true,
    "includeCompetitorDetails": true
}
```

Or with Semrush pages copied from your browser:

```json
{
    "startUrls": [
        { "url": "https://www.semrush.com/website/stripe.com/overview/" },
        { "url": "https://www.semrush.com/trending-websites/us/finance" }
    ],
    "maxItems": 500
}
```

| Field | Notes |
| --- | --- |
| `domains` | Domains to analyse: `stripe.com`, `www.stripe.com` or a full URL (`https://stripe.com/pricing`). The root domain is read (Semrush publishes one overview per root domain); what you typed is kept in `inputDomain`. |
| `startUrls` | Semrush pages: a domain page (`https://www.semrush.com/website/stripe.com/overview/`) or a top websites page (`https://www.semrush.com/trending-websites/us/finance`). |
| `location`, `locations` | Country of the top websites ranking: `us`, `fr`, `de`… or `global` for worldwide. Each country is crossed with each industry (max 500 country × category rankings per run, plus the ranking pages you paste in `startUrls`). |
| `topWebsitesCategories` | Industries of the ranking: `finance`, `banking`, `apparel-and-fashion`… or `all` for every website (default). |
| `enrichTopWebsites` | Also read the full domain overview of every ranked website: it is delivered as a domain row that keeps its rank. |
| `includeCompetitorDetails` | Add the authority, visits, ranks and engagement of up to 5 competitors of each domain (`competitorDetails`, one more page per domain, charged per competitor delivered: a domain with 3 competitors listed is charged 3, never 5). |
| `includeHistory` | Keep the monthly history columns (default on). Turn it off for lighter rows. |
| `maxItems` | Stop after this many rows for the whole run (`0` = unlimited). |
| `maxItemsPerQuery` | Cap for EACH ranking (country × industry): its top N. 0 = no per-list cap. |
| `excludeKeywords` | Skip the domains whose name contains one of these words: `casino`, `porn`. |
| `minAuthorityScore`, `minMonthlyVisits` | Keep only the domains above these figures (0 = no filter). Visits are the worldwide visits of the website; for a ranking row without `enrichTopWebsites`, its visits in the ranking's country. A domain without data is always kept, so you see it was not found. |
| `onlyNew`, `stateKey`, `resetState` | Monitoring: only the rows never delivered under this memory key; `resetState` forgets the memory. |
| Advanced | `proxyConfiguration` (Apify proxy by default, included in the price; the residential proxy is not available), `maxConcurrency`, `maxRequestsPerMinute`, `minRequestIntervalMs`, `maxRequestRetries`, `debugLog`. |

### 📦 Output

One domain row, from a real run on `stripe.com` (shortened: lists cut to their first entry, some columns left out):

```json
{
    "type": "domain",
    "id": "stripe.com",
    "url": "https://www.semrush.com/website/stripe.com/overview/",
    "domain": "stripe.com",
    "inputDomain": "https://stripe.com/pricing",
    "error": null,
    "found": true,
    "mainCountry": "US",
    "categories": ["Computer Software and Development", "Finance", "Information Technology", "Online Services"],
    "dataMonth": "2026-09-01",
    "globalRank": 145,
    "countryRank": 145,
    "countryRankCountry": "United States",
    "categoryRank": 3,
    "categoryRankName": "Finance",
    "authorityScore": 79,
    "visits": 232542245,
    "visitsChangePct": 11.79,
    "pagesPerVisit": 2.812,
    "avgVisitDurationSec": 498,
    "bounceRate": 0.6153,
    "desktopShare": 0.5397,
    "mobileShare": 0.4603,
    "organicTraffic": 4963905,
    "organicTrafficChangePct": -11.1,
    "paidTraffic": 192150,
    "backlinks": 141005083,
    "referringDomains": 732469,
    "googlePenaltyRisk": "Low",
    "topKeywords": [{ "keyword": "stripe login", "position": 1, "searchVolume": 135000, "cpc": 14.46, "trafficShare": 0.0708, "intents": ["commercial"] }],
    "competitors": [{ "domain": "gocardless.com", "relevance": 0.14, "visits": 3032397 }],
    "topCountry": "US",
    "topCountryVisits": 59382926,
    "topTrafficSource": "Direct",
    "aiTrafficVisits": 6801579,
    "visitsHistory": [{ "month": "2026-09-01", "value": 232542245 }],
    "scrapedAt": "2026-10-15T14:00:00.000Z"
}
```

You can download the dataset in various formats such as JSON, HTML, CSV or Excel.

Rows are saved as soon as they are read, so with several domains, several rankings or `enrichTopWebsites` they are not in the order you typed or in the order of the Semrush ranking: sort by `rank` (and `rankingCountry`) to get the ranking back.

#### All 65 fields

| Fields | What you get |
| --- | --- |
| `type`, `id`, `url` | `domain` or `topWebsite`; `stripe.com` (or `us/finance/stripe.com` for a ranked website); the Semrush page read |
| `domain`, `inputDomain`, `found`, `error` | root domain, what you typed, `false` + the reason when Semrush has no data (free row) |
| `brandName`, `mainCountry`, `categories`, `categorySlugs` | **Domain**: brand, `US`, `Finance`, `finance` |
| `dataMonth`, `semrushUpdatedAt`, `backlinksDataMonth` | month of the data, always its first day (`2026-09-01`), last update `2026-10-11`, month of the backlinks |
| `globalRank`, `globalRankTrend`, `countryRank`, `countryRankCountry`, `categoryRank`, `categoryRankName` | **Authority and rank**: `145`, `1` (going up), `145`, `United States`, `3`, `Finance` |
| `authorityScore` | Semrush Authority Score, 0-100 — `79` |
| `visits`, `visitsChangePct`, `pagesPerVisit`, `avgVisitDurationSec`, `bounceRate` | **Traffic**: `232542245`, `11.79` %, `2.812`, `498` s, `0.6153`. Worldwide figures for a domain row; a ranking row without `enrichTopWebsites` gets the ranking country's visits from the ranking |
| `desktopVisits`, `mobileVisits`, `desktopShare`, `mobileShare` | desktop and mobile visits, shares from 0 to 1 |
| `organicTraffic`, `organicTrafficChangePct`, `paidTraffic`, `paidTrafficChangePct` | **Search traffic** and its change in % |
| `backlinks`, `backlinksChangePct`, `referringDomains`, `referringDomainsChangePct`, `googlePenaltyRisk` | **Backlinks**: `141005083`, `-3.87`, `732469`, `4.94`, `Low` |
| `topKeywordsCountry`, `topKeywords` | the 5 keywords bringing most organic traffic: keyword, position, volume, CPC, share, intents by name (`commercial`, `informational`, `navigational`, `transactional`) |
| `competitors`, `competitorDetails` | **Competitors**: 3 domains with audience overlap and visits; with `includeCompetitorDetails`, up to 5 competitors (the Semrush competitors page lists 5) with their authority, visits, ranks (with the country of the country rank), pages per visit, duration, bounce rate |
| `topCountry`, `topCountryVisits`, `trafficByCountry` | **Countries**: `US`, `59382926`, the 5 top countries with their desktop / mobile share |
| `topTrafficSource`, `trafficSources`, `trafficDestinations` | **Traffic sources**: `Direct`; where visitors come from and where they go next, with share and visits — **desktop visits only** (Semrush's traffic journey does not count mobile), so `share` is a share of `desktopVisits`, not of `visits`. The host is cut from Semrush's channel: `google.com` + `channel: "Google organic"` (`channel` is null when there is none). Case is kept as Semrush writes it (`Direct` on a domain row, `direct` on a ranking row) |
| `aiTraffic`, `aiTrafficVisits` | **AI assistant traffic**: per assistant (visits, share, monthly history), and the sum of the assistants listed |
| `visitsHistory`, `searchTrafficHistory`, `deviceHistory`, `backlinksHistory`, `referringDomainsHistory` | **History**: 6 months, one entry per month: `{ month, value }` for `visitsHistory`, `backlinksHistory` and `referringDomainsHistory`, `{ month, organic, paid, total }` for `searchTrafficHistory`, `{ month, visits, desktopVisits, mobileVisits }` for `deviceHistory` (all empty with `includeHistory` off, as is the history of each `aiTraffic` assistant) |
| `rank`, `rankingCountry`, `rankingCategory`, `rankingCategoryName` | **Top websites**: `1`, `us`, `finance`, `Finance` |
| `rankingVisits`, `visitsMomChangePct`, `visitsYoyChangePct`, `rankingTotalVisits` | the ranking country's visits of the website (on every ranking row, with or without `enrichTopWebsites`), its change vs last month and last year in % (2 decimals), visits of all the websites of the ranking |
| `scrapedAt` | ISO timestamp |

### 💡 Tips

#### How to get more results

A ranking holds about 100 websites: to go wider, add countries in `locations` and industries in `topWebsitesCategories` — every country is crossed with every industry. Set `maxItems` to `0` to remove the overall cap.

#### How to reduce costs

The price is per row, so the levers are `maxItems`, `maxItemsPerQuery`, the filters and `onlyNew` for recurring runs (you never pay twice for the same row). `enrichTopWebsites` and `includeCompetitorDetails` are off by default: tick them only when you need the full overview of every ranked website or the figures of each competitor.

#### Several lists in one run

Fill `locations` and `topWebsitesCategories`: the Actor reads one ranking per country × industry (3 countries × 4 industries = 12 lists, up to 500 per run). The single `location` field is added to the list. Set `maxItemsPerQuery` to give every list its own cap: without it the first lists can use up the whole `maxItems` budget. Domains and rankings can be mixed in the same run.

#### Monitoring: only the new rows

Tick **Only new rows** (`onlyNew`) and schedule the Actor. The first run returns everything; each later run skips the rows already delivered: they are not saved and not charged. The memory lives in a named key-value store of your account (`semrush-seo-scraper-seen`, up to 150,000 rows per key) and is only updated with rows that really reached the dataset, so a failed run never hides anything. On a ranking with a cap (`maxItemsPerQuery`, or `maxItems`), the schedule watches that top N: each run returns the newcomers of the top N only, never the websites below it — a run where the top N has not changed returns nothing and charges nothing. Give each schedule its own `stateKey`, and tick `resetState` once to start over. Semrush updates its data once a month: a monthly schedule with `resetState` gives a fresh snapshot each month.

### 🔌 Integrations and API

Call the Actor via the Apify API, the JavaScript or Python clients, or connect it with integrations and webhooks (Make, Zapier, n8n, Google Sheets, Slack, Airtable…). The dataset can be fetched as JSON or CSV from any tool.

### 🤖 Use with AI agents (MCP)

AI agents (Claude, ChatGPT, Cursor…) can find and run this Actor through the [Apify MCP server](https://mcp.apify.com), billed to their Apify account like any run. It returns one row per domain overview or per ranked website from Semrush. Actor id: `nice_dev/semrush-seo-scraper`; MCP server with this Actor only: `https://mcp.apify.com/?tools=fetch-actor-details,nice_dev/semrush-seo-scraper`.

Smallest input, for a cheap first call:

```json
{
    "domains": ["stripe.com"],
    "maxItems": 10
}
```

Key output fields: `domain`, `authorityScore`, `visits`, `globalRank`, `organicTraffic`, `backlinks`, `topKeywords`, `competitors`, `aiTraffic`.

Cost: pay per row delivered, see the pricing section above. Cap each call with `maxItems` and, through the API, with the run option `maxTotalChargeUsd`.

### ❓ FAQ

#### Is it legal to scrape Semrush?

The Actor only reads what Semrush shows publicly to any anonymous visitor on its free website pages. It logs in to nothing and solves no captcha. The data are figures about websites, not about people. You are responsible for using the data in compliance with Semrush's Terms of Use and applicable law. This Actor is not affiliated with Semrush.

#### Does it need a login, a Semrush account or a proxy?

No login and no Semrush subscription. The proxy is included in the price: leave the default setting (the residential proxy is not available). A request the site turns away is retried at once on a new proxy session, up to 10 times on top of the retries (without a proxy, after a pause of 5 seconds, doubled at each retry up to 150 seconds).

#### Why does my domain come back with `found: false`?

Semrush publishes no overview for that domain (too little traffic, or too new). A subdomain you paste is read as its root domain (`docs.stripe.com` → `stripe.com`), so it never ends up here. The row tells you so in `error` and is not charged.

#### Which country are the keywords from?

The 5 top keywords are those of the domain's main country (`topKeywordsCountry`: `US` for stripe.com, `FR` for leboncoin.fr), as Semrush shows them; the country cannot be chosen.

#### Is the data safe to open in Excel or to show on a web page?

The texts are Semrush's own (domain names, keywords, category names), copied as they are. A keyword can begin with `-`, `+`, `=` or `@`: Excel and Google Sheets may read such a cell of a CSV file as a formula. The Actor leaves the text as it is, so that the JSON and the API give the real value: when you open a CSV, import these columns as text. On a web page, escape every field like any text from another site.

#### Known limitations

- Semrush shows the latest month only: older rankings and other keyword countries are not available.
- The free pages give 5 top keywords, 3 competitors and 5 countries per domain, not the full lists.
- `onlyNew` remembers row ids, not their content: a domain whose figures changed is not returned again (use `resetState` for a new snapshot).
- Two runs sharing the same `stateKey` at the same time may both return the same new row.

**A run that reaches its timeout** stops itself about 45 seconds before it: no new page is asked, what it read is saved and, with `onlyNew`, remembered, and the run ends *Succeeded* with "Stopped before the run's timeout". Resurrect it to go on from there, or give the next run a longer timeout (Run options).

**A run the platform stops without warning** (out of memory)

- Resurrect it: it goes on from where it stood at most a minute before the stop. What it had read since is read again, and the rows already saved are skipped: none is delivered or charged twice, and `maxItems` still counts them.
- With `onlyNew`, the memory is saved after each write (at most once every 2 seconds): resurrect the stopped run and the rows it had saved meanwhile join the memory; leave it stopped for good, and the next run may return the rows of its last 2 seconds once more.

#### Something doesn't work?

The last line of the log counts the rows saved and filtered out, and the requests that failed after every retry. Those requests are listed, with the reason, in the `FAILED_REQUESTS` record of the run's key-value store. A run that saved nothing and had failed requests fails, and its last message gives the cause.

If Semrush changes its pages, you are told instead of paying for blank rows. If the first 20 rows read all lack their domain, visits or Authority Score, the run saves nothing more, stops and fails, and its last message names the missing field: at most those first rows are charged. A run of fewer rows (a scheduled `onlyNew` run, a small **Max rows**) is checked at its end, from 3 rows: its rows are already saved, and the run fails with the missing field instead of ending green.

### 🛟 Support

Open an issue in the **Issues** tab with a link to your run: the run log and the `FAILED_REQUESTS` record of the key-value store show exactly which URLs failed and why.

# Actor input Schema

## `domains` (type: `array`):

Domains to analyse, one per line: `stripe.com`, `www.stripe.com` or a full URL (`https://stripe.com/pricing`) — the root domain is read. Semrush publishes one overview per root domain (a subdomain such as `docs.stripe.com` has no public page). Up to 5,000 domains.

## `startUrls` (type: `array`):

Semrush pages pasted from your browser: a domain page (`https://www.semrush.com/website/stripe.com/overview/`, also `/competitors/`) or a top websites page (`https://www.semrush.com/trending-websites/us/finance`). Added to the domains and lists below. Max 1 000 URLs.

## `location` (type: `string`):

Fill to get the Semrush ranking of the most visited websites (about 100 per country and category, latest month). Leave empty for domains only.

## `locations` (type: `array`):

More country codes (`us`, `fr`, `de`…, or `global`), added to the country above. Each country is crossed with each category below (500 country × category rankings per run at most, plus the ranking pages you paste in `startUrls`).

## `topWebsitesCategories` (type: `array`):

Industry categories of the ranking (`all` = every website). Empty = `all`.

## `enrichTopWebsites` (type: `boolean`):

Also read the full domain overview of every website found in the rankings (one more page per website, each delivered and billed as a domain row).

## `maxItems` (type: `integer`):

Maximum number of rows (domains + top websites) for the whole run. 0 = no limit.

## `maxItemsPerQuery` (type: `integer`):

Maximum number of websites taken from each country × category ranking (top first). 0 = the whole list (about 100). With Only new rows, it is the top N watched: each run delivers the newcomers of that top N only.

## `includeCompetitorDetails` (type: `boolean`):

Read the Semrush competitors page of each domain: up to 5 competitors with authority score, visits, ranks, pages per visit, time on site and bounce rate. One more page per domain; billed per competitor delivered (up to 5 per domain).

## `includeHistory` (type: `boolean`):

Keep the monthly history arrays (6 months of visits, organic and paid traffic, backlinks, referring domains, devices, AI traffic). Free — off only to get lighter rows.

## `excludeKeywords` (type: `array`):

Skip any domain whose name contains one of these words (case-insensitive), e.g. `casino`, `porn`. Applied to domains and top websites.

## `minAuthorityScore` (type: `integer`):

Keep only domains whose Semrush Authority Score (0-100) is at least this value. 0 = no filter. Applied to domain rows.

## `minMonthlyVisits` (type: `integer`):

Keep only domains and top websites with at least this many estimated visits in the last month: the worldwide visits of the website; for a ranking row without `enrichTopWebsites`, its visits in the ranking's country. 0 = no filter.

## `onlyNew` (type: `boolean`):

Skip domains and top websites already delivered by a previous run with the same memory key (monitoring a ranking: only newcomers are delivered). With a cap (Max top websites per list, Max rows), a ranking is watched on its top N only: a run delivers the newcomers of that top N, never the websites below it.

## `stateKey` (type: `string`):

Name of the memory used by **Only new rows**. Give each schedule / task its own key (e.g. `top-fr-finance`) so that they do not share their memory. Letters, digits, `-` and `_`.

## `resetState` (type: `boolean`):

Forget what previous runs delivered before this run starts.

## `proxyConfiguration` (type: `object`):

Apify Proxy or your own proxies. Keep the default: it is included in the price. The residential Apify proxy is not available in this Actor.

## `maxConcurrency` (type: `integer`):

Pages read in parallel.

## `maxRequestsPerMinute` (type: `integer`):

Upper bound of the request rate. With Competitor details, the competitors page of a domain counts as one more request.

## `minRequestIntervalMs` (type: `integer`):

Minimum delay in milliseconds between two requests.

## `maxRequestRetries` (type: `integer`):

Retries per request before it is marked as failed. Behind a proxy, a request the site turns away is also retried on a new proxy session up to 10 times without using up these retries.

## `debugLog` (type: `boolean`):

Verbose log.

## Actor input object example

```json
{
  "domains": [
    "stripe.com",
    "notion.so",
    "leboncoin.fr"
  ],
  "startUrls": [],
  "locations": [],
  "topWebsitesCategories": [
    "all"
  ],
  "enrichTopWebsites": false,
  "maxItems": 1000,
  "maxItemsPerQuery": 0,
  "includeCompetitorDetails": false,
  "includeHistory": true,
  "excludeKeywords": [],
  "minAuthorityScore": 0,
  "minMonthlyVisits": 0,
  "onlyNew": false,
  "stateKey": "default",
  "resetState": false,
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "maxConcurrency": 8,
  "maxRequestsPerMinute": 240,
  "minRequestIntervalMs": 0,
  "maxRequestRetries": 5,
  "debugLog": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "stripe.com",
        "notion.so",
        "leboncoin.fr"
    ],
    "topWebsitesCategories": [
        "all"
    ],
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("nice_dev/semrush-seo-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "domains": [
        "stripe.com",
        "notion.so",
        "leboncoin.fr",
    ],
    "topWebsitesCategories": ["all"],
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("nice_dev/semrush-seo-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "stripe.com",
    "notion.so",
    "leboncoin.fr"
  ],
  "topWebsitesCategories": [
    "all"
  ],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call nice_dev/semrush-seo-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nice_dev/semrush-seo-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/kYMDdjsbI4QKYBq04/builds/AeZfl4mOhs3VI6hZ3/openapi.json