# Semrush Website Traffic Stats Scraper (`automation-lab/semrush-website-traffic-stats-scraper`) Actor Extract public Semrush traffic stats for bulk domain lists: visits, engagement, organic and paid traffic, countries, keywords, and competitors. - **URL**: https://apify.com/automation-lab/semrush-website-traffic-stats-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** SEO tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Semrush Website Traffic Stats Scraper Extract public Semrush website traffic stats for a bulk list of domains. This actor reads anonymous Semrush `/website/{domain}/overview/` pages and turns visible overview metrics into structured dataset records. Use it to compare competitors, qualify SEO prospects, enrich lead lists, and monitor market traffic signals without copying numbers by hand. ### What does Semrush Website Traffic Stats Scraper do? It fetches public Semrush website overview pages and extracts the metrics that are visible in the page HTML. - 🌐 Total website visits - 📈 Visit history - 🔎 Organic and paid traffic estimates - 🌍 Top countries and traffic share - 🧭 Engagement metrics such as pages per visit and bounce rate - 🏁 Similar competitors shown by Semrush - 🔑 Top organic keywords when visible - 🤖 AI traffic sources when Semrush exposes them publicly ### Who is it for? This scraper is designed for practical SEO and growth workflows. - SEO agencies auditing prospect lists - Growth teams comparing competitors - Demand generation teams enriching accounts - Competitive intelligence analysts building market snapshots - Content teams prioritizing websites by search potential - Investors and operators checking public traffic trends ### Why use this actor? Manual Semrush checks are slow when you have dozens or hundreds of domains. This actor accepts a list of domains and returns one clean record per website. You can export the dataset to JSON, CSV, Excel, Google Sheets, or consume it through the Apify API. ### What data can you extract? | Field | Description | | --- | --- | | `domain` | Domain that was checked | | `semrushUrl` | Public Semrush overview URL | | `totalVisits` | Estimated visits from the public overview | | `pagesPerVisit` | Estimated pages per visit | | `averageVisitDurationSeconds` | Average visit duration in seconds | | `bounceRate` | Estimated bounce rate | | `organicVisits` | Estimated organic search visits | | `paidVisits` | Estimated paid search visits | | `topCountries` | Country traffic and share breakdown | | `topOrganicKeywords` | Visible top keyword metrics | | `competitors` | Similar domains from Semrush | | `aiTrafficSources` | Visible AI-source traffic signals | | `visitsHistory` | Historical visit estimates | | `searchTrafficHistory` | Organic and paid trend points | | `dataFound` | Whether public metrics were found | | `error` | Extraction warning or request error | ### How much does it cost to extract Semrush website traffic stats? The actor uses pay-per-event pricing. - A small start event is charged once per run. - One item event is charged for each domain record saved. - You control spend with `maxItems` and the number of domains in the input. Start with 5-10 domains, inspect the output, then scale to larger batches. ### Input You can provide domains directly: ```json { "domains": ["google.com", "microsoft.com", "apple.com"], "maxItems": 10 } ```` You can also provide Semrush overview URLs: ```json { "startUrls": [ { "url": "https://www.semrush.com/website/google.com/overview/" } ], "maxItems": 10 } ``` ### Input fields #### `domains` A list of domains such as `example.com`. Do not include login-only Semrush project URLs. #### `startUrls` Optional request-list input for Semrush overview URLs or normal website URLs. The actor extracts the domain from each URL. #### `maxItems` Maximum unique domains to process. This is a safety cap for first runs and scheduled jobs. ### Output example ```json { "domain": "google.com", "semrushUrl": "https://www.semrush.com/website/google.com/overview/", "totalVisits": 101172727926, "organicVisits": 5235637014, "paidVisits": 21228492, "pagesPerVisit": 5.6419, "bounceRate": 0.3669, "topCountries": [{ "countryCode": "US", "countryName": "United States" }], "dataFound": true, "scrapedAt": "2026-07-02T00:00:00.000Z" } ``` ### How to run it 1. Open the actor on Apify. 2. Paste your domains into the `Domains` input. 3. Set `Maximum domains` for the run. 4. Start the actor. 5. Download results from the dataset tab. ### Tips for best results - Use root domains like `example.com` instead of long URLs. - Keep the first run small to verify the exact fields visible for your domains. - Some domains may have limited public Semrush data; check `dataFound` and `error`. - Schedule repeat runs if you need periodic competitive snapshots. ### Integrations Use the output in common SEO workflows: - Push high-traffic competitors into a BI dashboard. - Enrich CRM accounts with public traffic estimates. - Join `topCountries` with market priority lists. - Send domains with rising organic traffic to a content research queue. - Compare paid and organic traffic mix for acquisition planning. ### API usage with Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/semrush-website-traffic-stats-scraper').call({ domains: ['google.com', 'microsoft.com'], maxItems: 2, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` ### API usage with Python ```python from apify_client import ApifyClient import os client = ApifyClient(os.environ['APIFY_TOKEN']) run = client.actor('automation-lab/semrush-website-traffic-stats-scraper').call(run_input={ 'domains': ['google.com', 'microsoft.com'], 'maxItems': 2, }) items = client.dataset(run['defaultDatasetId']).list_items().items print(items) ``` ### API usage with cURL ```bash curl -X POST "https://api.apify.com/v2/acts/automation-lab~semrush-website-traffic-stats-scraper/runs?token=$APIFY_TOKEN" \ -H 'Content-Type: application/json' \ -d '{"domains":["google.com","microsoft.com"],"maxItems":2}' ``` ### MCP usage Connect through Apify MCP with this tool URL: ```text https://mcp.apify.com/?tools=automation-lab/semrush-website-traffic-stats-scraper ``` Add it to Claude Code: ```bash claude mcp add apify-semrush-traffic "https://mcp.apify.com/?tools=automation-lab/semrush-website-traffic-stats-scraper" ``` Or add an MCP server JSON config: ```json { "mcpServers": { "apify-semrush-traffic": { "url": "https://mcp.apify.com/?tools=automation-lab/semrush-website-traffic-stats-scraper" } } } ``` Example prompts: - "Use the Apify Semrush traffic scraper to analyze google.com, microsoft.com, and apple.com, then summarize organic vs paid traffic." - "Run the MCP Semrush Website Traffic Stats Scraper for this prospect list and rank the domains by visits." - "Compare top countries for these SaaS competitors using the Apify actor output." ### Scheduling Create a scheduled task to monitor the same domain set weekly or monthly. Export each dataset to your warehouse or spreadsheet to track trend changes over time. ### Data freshness Semrush public overview pages display Semrush's current public estimates. The actor records the dates visible in the page, such as `trafficDate` and `organicTrafficDate`, so you can understand the period behind the metrics. ### Reliability notes The actor uses HTTP requests rather than a browser. This keeps runs fast and inexpensive. If Semrush changes the public page structure, records may return `dataFound: false` with an explanatory `error` field instead of failing the whole run. ### FAQ #### Why is `dataFound` false? The domain may not have public metrics, Semrush may have changed the page, or the request may have returned a limited page. Check `httpStatus`, `pageTitle`, and `error`. #### Can I use Semrush account-only data? No. This actor is for public anonymous Semrush overview pages only. It does not log in and does not use private Semrush projects. ### Legality This actor extracts public web page data. Make sure your use case complies with applicable laws, Semrush terms, and privacy requirements. Do not use the output for unlawful profiling or spam. ### Related scrapers Other automation-lab actors can complement this workflow: - Google Search Results Scraper for SERP visibility checks - Website Contact Finder for lead enrichment - Website Technology Lookup actors for account qualification - SEO audit and metadata scrapers for on-page checks ### Changelog - Initial version: public Semrush overview extraction for traffic, engagement, geography, keywords, competitors, and trend fields. ### Limitations - Metrics are Semrush estimates, not first-party analytics. - Public pages can expose different fields for different domains. - The actor cannot bypass private account walls. - Very large runs should be tested gradually. ### Support If a field stops appearing, include a sample domain and run ID when requesting support. The `dataFound`, `httpStatus`, and `error` fields help diagnose page changes quickly. # Actor input Schema ## `domains` (type: `array`): Domain names to look up on public Semrush website overview pages. Use domains like google.com, microsoft.com, or apple.com. ## `startUrls` (type: `array`): Optional Semrush /website/{domain}/overview/ URLs or regular domain URLs. Domains from these URLs are merged with the Domains list. ## `maxItems` (type: `integer`): Maximum number of unique domains to process in this run. Keep the first run small, then scale up for bulk prospect lists. ## Actor input object example ```json { "domains": [ "google.com", "microsoft.com", "apple.com" ], "startUrls": [ { "url": "https://www.semrush.com/website/google.com/overview/" } ], "maxItems": 10 } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": [ "google.com", "microsoft.com", "apple.com" ], "startUrls": [ { "url": "https://www.semrush.com/website/google.com/overview/" } ], "maxItems": 10 }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/semrush-website-traffic-stats-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": [ "google.com", "microsoft.com", "apple.com", ], "startUrls": [{ "url": "https://www.semrush.com/website/google.com/overview/" }], "maxItems": 10, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/semrush-website-traffic-stats-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": [ "google.com", "microsoft.com", "apple.com" ], "startUrls": [ { "url": "https://www.semrush.com/website/google.com/overview/" } ], "maxItems": 10 }' | apify call automation-lab/semrush-website-traffic-stats-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/semrush-website-traffic-stats-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Semrush Website Traffic Stats Scraper", "description": "Extract public Semrush traffic stats for bulk domain lists: visits, engagement, organic and paid traffic, countries, keywords, and competitors.", "version": "0.1", "x-build-id": "oot4gPYU53GIea96p" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~semrush-website-traffic-stats-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-semrush-website-traffic-stats-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~semrush-website-traffic-stats-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-semrush-website-traffic-stats-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~semrush-website-traffic-stats-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-semrush-website-traffic-stats-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "domains": { "title": "Domains", "type": "array", "description": "Domain names to look up on public Semrush website overview pages. Use domains like google.com, microsoft.com, or apple.com.", "items": { "type": "string" } }, "startUrls": { "title": "Semrush overview URLs", "type": "array", "description": "Optional Semrush /website/{domain}/overview/ URLs or regular domain URLs. Domains from these URLs are merged with the Domains list.", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "title": "URL of a web page", "format": "uri" } } } }, "maxItems": { "title": "Maximum domains", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum number of unique domains to process in this run. Keep the first run small, then scale up for bulk prospect lists.", "default": 10 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```