Semrush Domain Overview Scraper

Scrape the Semrush website for public domain overview SEO metrics, including traffic, rank, authority, backlinks, keywords, country shares, and competitor summaries.

What does Semrush Domain Overview Scraper do?

Semrush Domain Overview Scraper is an SEO data scraper for the Semrush website and public domain overview pages. Add domains, run the actor, and export structured traffic, ranking, authority, backlink, keyword, country-share, and competitor summary fields as JSON, CSV, Excel, or via API for SEO monitoring and competitive intelligence.

Use cases: who uses it

• SEO teams tracking traffic and authority snapshots for client domains.
• Growth teams benchmarking a product against competitors.
• Agencies preparing recurring domain-performance reports.
• Data teams feeding public competitive-intelligence metrics into dashboards.

Input example

{
  "domains": ["openai.com", "github.com", "stripe.com"],
  "maxItems": 3,
  "maxConcurrency": 2,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["SHADER"] }
}

Output example

{
  "input": "openai.com",
  "domain": "openai.com",
  "url": "https://www.semrush.com/website/openai.com/overview/",
  "status": "success",
  "reportingPeriod": "2026-06-01",
  "visits": { "raw": "508002719", "value": 508002719, "diffPercent": -0.0945 },
  "authorityScore": { "raw": "99", "value": 99, "diffPercent": null },
  "referringDomains": { "raw": "423708", "value": 423708, "diffPercent": 0.0496 },
  "backlinks": { "raw": "64639384", "value": 64639384, "diffPercent": 0.015 },
  "topCountries": [{ "country": "United States", "share": 0.1397, "visits": 70981896 }],
  "error": null
}

Input recipes

Bulk domain snapshot

{
  "domains": ["openai.com", "github.com", "stripe.com"],
  "maxItems": 3,
  "maxConcurrency": 2
}

Competitor benchmark batch

{
  "domains": ["notion.so", "clickup.com", "asana.com", "monday.com"],
  "maxItems": 4,
  "maxConcurrency": 2
}

Input settings

Field	Type	Description
domains	array of strings	Bare domains or URLs to normalize and fetch.
maxItems	integer	Maximum unique domains to process.
maxConcurrency	integer	Conservative parallelism control; keep low for reliable public pages.
proxyConfiguration	object	Optional Apify proxy settings. SHADER/datacenter is the default starting route.
includeRawHtmlSnippet	boolean	Adds a short troubleshooting snippet to each row.
saveDebugArtifacts	boolean	Saves run diagnostics to the key-value store.

Output fields

Field	Description
input, domain, url	Original input, normalized domain, and public Semrush overview URL.
status, error	success, no_public_metrics, not_found, blocked, failed, or invalid_input.
fetchedAt, reportingPeriod	Collection time and visible Semrush reporting period when available.
title, description	Public page metadata.
globalRank, countryRank, categoryRank	Ranking metrics when present.
visits, pagesPerVisit, avgVisitDuration, bounceRate	Traffic and engagement summaries.
organicSearchTraffic, paidSearchTraffic, organicKeywords	Public search summary metrics.
authorityScore, referringDomains, backlinks	Authority and backlink summary counts.
topCountries, topCategories, competitors	Arrays of public breakdowns when present.

Metric objects include raw, normalized numeric value when reliable, and diffPercent when Semrush exposes a change percentage.

Pricing

This actor uses pay-per-event pricing:

• Start: $0.005 per run.
• Successful domain overview: tiered by Apify plan.

Tier	Price per successful row
Free	$0.000093291
Bronze	$0.000081123
Silver	$0.000063276
Gold	$0.000048674
Platinum	$0.000032449
Diamond	$0.000022714

Rows with invalid_input, not_found, blocked, failed, or no_public_metrics statuses are saved for diagnostics but are not charged as successful domain overview results.

Tips and limits

• Start with 3-10 domains to validate your workflow.
• Use bare domains such as example.com; URLs are accepted and normalized.
• Keep concurrency low because public Semrush pages may rate-limit aggressive bulk traffic.
• This actor extracts public overview-page summaries only. It does not log in to Semrush or export private account data.

API usage

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/semrush-domain-overview-scraper').call({
  domains: ['openai.com', 'github.com'],
  maxItems: 2
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient('MY_APIFY_TOKEN')
run = client.actor('fetch_cat/semrush-domain-overview-scraper').call(run_input={
    'domains': ['openai.com', 'github.com'],
    'maxItems': 2,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~semrush-domain-overview-scraper/runs?token=MY_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"domains":["openai.com","github.com"],"maxItems":2}'

MCP and agents

Use the Apify MCP server with this actor tool enabled:

$claude mcp add apify 'https://mcp.apify.com/?tools=fetch_cat/semrush-domain-overview-scraper'

JSON configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=fetch_cat/semrush-domain-overview-scraper"
    }
  }
}

Example prompts:

• "Run a Semrush overview snapshot for openai.com, github.com, and stripe.com."
• "Compare public Semrush authority and backlink metrics for these competitor domains."

Related actors

• Google Maps Lead Finder
• Yelp Scraper

FAQ

Do I need a Semrush account?
No. V1 reads public Semrush domain overview pages only.

Will every domain have all fields?
No. Semrush may omit metrics for smaller or unavailable domains. The actor returns nullable fields and a status per domain.

Can one failed domain fail the whole run?
No. Per-domain failures are saved as rows and the actor continues with remaining domains.

Support

When opening an issue, include:

• Actor run ID or run URL.
• Full input JSON.
• Expected output.
• Actual output or status/error row.
• Reproducible public URL, for example a Semrush domain overview URL, if one specific domain fails.