# SEMrush Scraper — Traffic, Authority Score & AI Visits (`convertfleetdotonline/semrush-domain-seo-metrics`) Actor Extract 30+ SEO metrics for any domain in seconds: authority score, website traffic, backlinks, keywords with CPC, competitors, AI/ChatGPT referral traffic & 6-month history. HTTP-only (no browser) = fast & cheap. No SEMrush account needed. - **URL**: https://apify.com/convertfleetdotonline/semrush-domain-seo-metrics.md - **Developed by:** [Hasnain Nisar](https://apify.com/convertfleetdotonline) (community) - **Categories:** SEO tools, Automation, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## SEMrush Scraper — Website Traffic, Authority Score & SEO Metrics (with AI Traffic) Check **domain authority, website traffic, backlinks, top keywords and competitors** for any domain — **no SEMrush subscription required**. This scraper reads SEMrush's public website directory and returns **30+ clean, structured SEO metrics per domain** in seconds. Unlike browser-based scrapers, this Actor is **HTTP-only**: no headless Chrome, no rendering wait. A typical domain finishes in **3–6 seconds** and large batches run in parallel. ### ⚡ Why this SEMrush scraper? - 🚀 **Fast & lightweight** — pure HTTP extraction, ~3–6s per domain (browser scrapers need 35–60s) - 🤖 **AI traffic data** — visits referred by **ChatGPT, Gemini and other AI assistants**, the metric every GEO / AI-visibility report needs in 2026 - 📈 **6-month history** — traffic history AND backlink history, not just a snapshot - 💰 **Keyword CPC + search intent** included with every top keyword - 🔗 **Top referral sources** with share and visits (including Direct share) - 🌍 **Per-country device split** (desktop vs mobile) on top of the global split - 🆓 **No SEMrush account, no API key, no login** — public data only ### 📊 Data you get per domain | Group | Fields | |---|---| | Authority & rank | `domain_authority` (0–100), `global_rank`, `country_rank` + `country`, `google_penalty_risk`, `industry_category` | | Traffic & engagement | `monthly_visits`, `monthly_visits_change_percent`, `bounce_rate`, `pages_per_visit`, `avg_visit_duration_seconds` | | Search traffic | `organic_search_visits`, `organic_traffic_change_percent`, `paid_search_visits`, `paid_traffic_change_percent` | | Sources | `direct_visits_percent`, `top_traffic_sources[]` (source, share %, visits) | | 🤖 AI visibility | `ai_traffic[]` — visits from ChatGPT, Gemini & other AI assistants with share % | | Backlinks | `total_backlinks`, `backlinks_change_percent`, `referring_domains`, `referring_domains_change_percent` | | Keywords | `organic_keywords[]` — keyword, position, search volume, **CPC**, **intent**, traffic share | | Competitors | `competitors[]` — domain, relevance score, monthly visits | | Geography & devices | `top_countries[]` with traffic %, visits and per-country desktop/mobile split; overall `desktop_traffic_percent` / `mobile_traffic_percent` | | History | `visits_history[]` and `backlinks_history[]` (up to 6 months) | ### 🔧 Input ```json { "domains": ["amazon.com", "shopify.com", "zapier.com"], "maxConcurrency": 5, "proxyConfiguration": { "useApifyProxy": true } } ```` - **`domains`** *(required)* — up to 500 domains per run. Full URLs are fine; they are normalized to root domains automatically. - **`maxConcurrency`** — parallel fetches (default 5, max 10). - **`proxyConfiguration`** — Apify datacenter proxies (free) work well by default. ### 📤 Output example ```json { "domain": "make.com", "data_month": "2026-06", "domain_authority": 57, "global_rank": 6414, "country_rank": 6856, "country": "United States", "industry_category": "Computer Software and Development", "monthly_visits": 8367047, "monthly_visits_change_percent": -8.69, "bounce_rate": 39.46, "pages_per_visit": 7.05, "avg_visit_duration_seconds": 772, "organic_search_visits": 637690, "paid_search_visits": 124775, "direct_visits_percent": 84.28, "total_backlinks": 2050718, "referring_domains": 38512, "ai_traffic": [ { "source": "chatgpt.com", "name": "ChatGPT", "visits": 63665, "share_percent": 0.77 }, { "source": "gemini.google.com", "name": "Gemini", "visits": 21643, "share_percent": 0.26 } ], "organic_keywords": [ { "keyword": "make ai", "position": 1, "search_volume": 4400, "cpc": 3.15, "intents": "2" } ], "competitors": [ { "domain": "n8n.io", "relevance": 0.08, "monthly_visits": 6864336 } ], "top_countries": [ { "country": "United States", "traffic_percent": 19.36, "visits": 1619950, "desktop_share_percent": 48.96 } ], "visits_history": [ { "date": "2026-06-01", "visits": 8367047 }, { "date": "2026-05-01", "visits": 9163168 } ] } ``` ### 💡 Use cases - **Competitor analysis** — compare authority score, traffic trends, keyword overlap and traffic sources across your whole market in one run - **SEO audits & client reporting** — drop live domain KPIs into audit headers and monthly reports - **AI visibility / GEO tracking** — monitor how much traffic competitors already get from ChatGPT and Gemini - **Domain due diligence** — vet domains before buying: authority, backlink trend, penalty risk, traffic history - **Link prospecting & lead scoring** — qualify outreach lists by authority score and referring domains - **Market research** — map industry leaders, geography and device mix in any niche ### ❓ FAQ **Do I need a SEMrush account or API key?** No. The Actor reads SEMrush's public website directory pages only. **How fresh is the data?** SEMrush refreshes its public directory roughly monthly. Each result includes `data_month` so you always know which month you're looking at. **Why are some fields `null`?** Low-traffic or brand-new domains often have no public data — SEMrush simply hasn't profiled them. You still get an item with the domain and whatever is available. **Why does a big site return an error?** A few domains (e.g. zapier.com) have opted out of SEMrush's public directory. Those return a clear `error` item — `"Domain not found in the SEMrush public website directory"` — instead of misleading nulls. **Can I check subdomains?** Root domains work best (`example.com`). Subdomains only return data when SEMrush publishes a profile for them. **How fast is it? What does it cost?** \~3–6 seconds per domain with default settings. Because there is no browser, compute cost per domain is a fraction of Playwright-based scrapers. **Integrations?** Works with the Apify API, scheduler, webhooks and all Apify integrations (Make, Zapier, Google Sheets, Slack). Example API call: ```bash curl -X POST "https://api.apify.com/v2/acts//run-sync-get-dataset-items?token=" \ -H "Content-Type: application/json" \ -d '{"domains": ["amazon.com"]}' ``` ### ⚖️ Disclaimer This Actor is not affiliated with, endorsed by, or sponsored by Semrush Inc. "SEMrush" is a trademark of its respective owner, used here only to describe the data source. The Actor extracts publicly accessible data only. You are responsible for how you use the data and for compliance with applicable laws and terms of service. # Actor input Schema ## `domains` (type: `array`): List of domains to pull SEMrush metrics for (e.g. amazon.com, shopify.com). Root domains work best; URLs are normalized automatically. ## `maxConcurrency` (type: `integer`): How many domains to fetch in parallel. ## `proxyConfiguration` (type: `object`): Proxy settings. Apify datacenter proxies (included free) work well. ## Actor input object example ```json { "domains": [ "amazon.com", "shopify.com", "zapier.com" ], "maxConcurrency": 5, "proxyConfiguration": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": [ "make.com" ], "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("convertfleetdotonline/semrush-domain-seo-metrics").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": ["make.com"], "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("convertfleetdotonline/semrush-domain-seo-metrics").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": [ "make.com" ], "proxyConfiguration": { "useApifyProxy": true } }' | apify call convertfleetdotonline/semrush-domain-seo-metrics --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=convertfleetdotonline/semrush-domain-seo-metrics", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "SEMrush Scraper — Traffic, Authority Score & AI Visits", "description": "Extract 30+ SEO metrics for any domain in seconds: authority score, website traffic, backlinks, keywords with CPC, competitors, AI/ChatGPT referral traffic & 6-month history. HTTP-only (no browser) = fast & cheap. No SEMrush account needed.", "version": "1.0", "x-build-id": "zRbzAlyzhNHylkkbf" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/convertfleetdotonline~semrush-domain-seo-metrics/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-convertfleetdotonline-semrush-domain-seo-metrics", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/convertfleetdotonline~semrush-domain-seo-metrics/runs": { "post": { "operationId": "runs-sync-convertfleetdotonline-semrush-domain-seo-metrics", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/convertfleetdotonline~semrush-domain-seo-metrics/run-sync": { "post": { "operationId": "run-sync-convertfleetdotonline-semrush-domain-seo-metrics", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "domains" ], "properties": { "domains": { "title": "Domains to analyze", "minItems": 1, "maxItems": 500, "type": "array", "description": "List of domains to pull SEMrush metrics for (e.g. amazon.com, shopify.com). Root domains work best; URLs are normalized automatically.", "items": { "type": "string", "pattern": "^(https?://)?(www\\.)?([a-zA-Z0-9]([a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\\.)+[a-zA-Z]{2,}(/.*)?$" } }, "maxConcurrency": { "title": "Max concurrency", "minimum": 1, "maximum": 10, "type": "integer", "description": "How many domains to fetch in parallel.", "default": 5 }, "proxyConfiguration": { "title": "Proxy configuration", "type": "object", "description": "Proxy settings. Apify datacenter proxies (included free) work well." } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```