# Semrush Scraper (`automly/semrush-scraper`) Actor Get Semrush data for any list of websites: Authority Score, organic and paid traffic, backlinks, referring domains, ranks, top keywords, competitors and AI traffic. Plus top websites by country and industry. - **URL**: https://apify.com/automly/semrush-scraper.md - **Developed by:** [Automly](https://apify.com/automly) (community) - **Categories:** SEO tools - **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.00 / 1,000 website overviews This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Semrush Scraper ### What is Semrush Scraper? **Semrush Scraper** lets you **get Semrush website data for any list of domains**: Authority Score, monthly visits, organic and paid search traffic, backlinks and referring domains, global, country and category rank, top organic keywords with search volume, CPC and intent, main competitors, top countries, traffic sources, AI assistant traffic (ChatGPT, Gemini and more) and month-by-month history. It also lists **the top 100 websites of any country and industry**, ranked by visits. Add your domains, click **Start**, and download one row per website as Excel, CSV or JSON. - ⚡ **Fast:** 100 websites in under a minute - 📋 **Bulk lists:** paste hundreds of domains or full links at once; repeats are read once - 📊 **One row per website:** headline numbers in their own columns, lists and history next to them - 🏆 **Top websites:** the 100 most visited websites of 240 countries and 110 industries - 🔓 **No login:** no Semrush account, subscription or API key needed ### What can Semrush Scraper do? - **Check Authority Score in bulk** for any list of websites, with backlinks and referring domains - **Estimate website traffic**: monthly visits, organic and paid search traffic and their change from the month before - **Get Semrush ranks**: global rank, rank in the website's main country and rank in its category - **See a website's top organic keywords** with position, search volume, CPC, traffic share and search intent - **Find a website's competitors** with their relevance and monthly visits - **See where traffic comes from and goes to**: direct, search engines and referring websites, and the sites visitors open next - **Measure AI traffic**: visits from ChatGPT, Gemini, Perplexity and other AI assistants - **List the top websites** of a country and industry (for example the top finance websites in the United States) - Export to Excel, CSV, JSON, HTML or XML, or send the data to Google Sheets, Make, Zapier and more ### What data can you extract from Semrush? | | | | |---|---|---| | 🛡️ Authority Score | 🔗 Backlinks and referring domains | ⚠️ Backlink penalty risk | | 👥 Monthly visits and change | 🔍 Organic search traffic | 💰 Paid search traffic | | 🏆 Global, country and category rank | 🗂️ Categories | 🌍 Top countries with shares | | 🔑 Top keywords, volume, CPC, intent | 🥊 Competitors | 🚦 Traffic sources and outgoing sites | | 🤖 AI assistant traffic | 📱 Desktop vs mobile | ⏱️ Visit duration, pages per visit, bounce rate | | 📈 Visits history | 📈 Search traffic history | 📈 Backlinks history | ### How to scrape Semrush 1. [Create a free Apify account](https://console.apify.com/sign-up). 2. Open **Semrush Scraper**. 3. Enter the websites to analyze, one per line: `nike.com`, `https://www.stripe.com/pricing` or a Semrush website link all work. 4. For top websites lists, pick one or more **Countries** and **Industries** (or leave the websites empty and use only the lists). 5. Click **Start**, then download the results as Excel, CSV, JSON or HTML. ### How much does it cost to scrape Semrush? You pay per result saved: one result per website, and one per website in a top websites list. Websites Semrush has no data for are not charged. See the **Pricing** tab for the price on your Apify plan, and set a maximum cost per run if you like: the run stops there. ### ⬇️ Input | Setting | What it does | |---|---| | **Websites** | Domains (`nike.com`), links (`https://www.nike.com/us/`) or Semrush website links, one per line. `www.`, pages and subdomains are read as the main website (`blog.hubspot.com` gives `hubspot.com`) | | **Countries** | Countries to list the top websites of, or **Worldwide**. Empty: no lists | | **Industries** | Industries to list for each country. **All industries** by default | | **Proxy configuration** | The default works for most runs | Overview data for three websites: ```json { "domains": ["nike.com", "stripe.com", "https://www.bbc.co.uk/"] } ``` The top finance and travel websites in the United States and the United Kingdom (400 rows): ```json { "topWebsitesCountries": ["us", "gb"], "topWebsitesIndustries": ["finance", "travel-and-tourism"] } ``` ### ⬆️ Output Every website is one row. Shares, rates and changes are percentages (a change can be negative), visits and ranks are whole numbers, durations are in seconds. A website row (lists shortened): ```json { "type": "domain", "domain": "nike.com", "input": "https://www.nike.com/x", "url": "https://www.semrush.com/website/nike.com/overview/", "brandName": "Nike", "mainCountry": "US", "categories": ["Retail", "Online Services", "Sporting Goods"], "dataMonth": "2026-09-15", "updatedAt": "2026-10-11", "authorityScore": 99, "googlePenaltyRisk": "Low", "globalRank": 446, "countryRank": 316, "countryRankCountry": "United States", "categoryRank": 1, "categoryRankName": "Sporting Goods", "visits": 83426980, "visitsChangePercent": -14.39, "pagesPerVisit": 4.5148, "avgVisitDuration": 382, "bounceRate": 49.08, "desktopShare": 32.77, "mobileShare": 67.23, "organicTraffic": 108777460, "organicTrafficChangePercent": -8.04, "paidTraffic": 1413275, "paidTrafficChangePercent": -37.94, "backlinks": 33718134, "backlinksChangePercent": -1.73, "referringDomains": 203695, "referringDomainsChangePercent": -0.19, "aiTraffic": 166612, "aiTrafficShare": 0.202, "topKeywordsCountry": "US", "topKeywords": [ {"keyword": "nike", "position": 1, "searchVolume": 3350000, "cpc": 0.73, "trafficShare": 8.98, "intents": ["commercial", "navigational"]}, {"keyword": "nike shoes", "position": 1, "searchVolume": 550000, "cpc": 0.7, "trafficShare": 1.47, "intents": ["informational", "transactional"]} ], "competitors": [ {"domain": "footlocker.com", "relevance": 45.0, "visits": 12257461}, {"domain": "dickssportinggoods.com", "relevance": 29.0, "visits": 33485983} ], "trafficByCountry": [ {"country": "US", "countryName": "United States", "visits": 30507483, "share": 36.57, "desktopShare": 34.96, "mobileShare": 65.04} ], "trafficSources": [ {"source": "Direct", "sourceType": null, "share": 46.22, "visits": 12635822, "previousVisits": 13437127}, {"source": "google.com", "sourceType": "Google organic", "share": 31.65, "visits": 8653037, "previousVisits": 8129281} ], "outgoingSites": [ {"domain": "okta.com", "share": 11.08, "visits": 259532, "previousVisits": 253914} ], "aiTrafficSources": [ {"source": "chatgpt.com", "name": "ChatGPT", "visits": 152625, "share": 0.185, "history": [{"month": "2026-09-01", "visits": 152625}, {"month": "2026-08-01", "visits": 111983}]} ], "visitsHistory": [{"month": "2026-09-01", "visits": 83426980, "desktopVisits": 27337690, "mobileVisits": 56089290}], "searchTrafficHistory": [{"month": "2026-09-15", "organic": 108777460, "paid": 1413275}], "backlinksHistory": [{"month": "2026-09-01", "backlinks": 33718134, "referringDomains": 203695}], "scrapedAt": "2026-10-28T10:16:36Z" } ``` A row of a top websites list: ```json { "type": "topWebsite", "rank": 1, "domain": "yahoo.com", "country": "us", "countryName": "United States", "industry": "finance", "industryName": "Finance", "dataMonth": "2026-09-01", "visits": 1133441981, "desktopVisits": 464513538, "mobileVisits": 668928443, "desktopShare": 40.98, "mobileShare": 59.02, "visitsChangePercent": -1.22, "visitsChangeYearPercent": -16.39, "topSource": "direct", "url": "https://www.semrush.com/website/yahoo.com/overview/", "scrapedAt": "2026-10-28T10:16:36Z" } ``` The **Websites** tab shows the headline numbers of every website side by side, the **Top websites** tab the ranked lists. The run report lists any website Semrush has no data for. ### How can I use Semrush data? - **Competitor analysis:** compare Authority Score, traffic, keywords and backlinks of your competitors in one table - **Link building and outreach:** sort prospect websites by Authority Score and traffic before you reach out - **Lead qualification:** size up companies by their website traffic and growth - **Market research:** list the top websites of an industry in any country and follow them month by month - **SEO reporting:** track your own and your clients' Authority Score, organic traffic and backlinks with a schedule - **AI search tracking:** see how much traffic websites get from ChatGPT, Gemini and other AI assistants ### Which websites have data? Most well-known websites are covered, including country sites such as `bbc.co.uk` or `nike.com.br`. Semrush does not show data for every website without a subscription, and many small websites have none: those are listed in the run report as not found, are not charged, and the run still finishes successfully. Smaller websites may have some sections empty (for example no competitors or keywords). Figures are Semrush's estimates for the month in `dataMonth`. ### Scrape more website and SEO data | Actor | What it gets | |---|---| | [Ahrefs Scraper](https://apify.com/automly/ahrefs-seo-scraper) | Domain Rating, backlinks, organic traffic and keyword ideas for any domain | | [Similarweb Scraper](https://apify.com/automly/similarweb-scraper) | Website traffic, ranks, traffic sources and similar sites from Similarweb | | [Domain Authority Checker](https://apify.com/automly/domain-authority-checker) | An authority score and referring domains for millions of websites | | [Website Email Scraper](https://apify.com/automly/website-email-scraper) | Email addresses from any website's contact, about and team pages | | [Google Ads Transparency Scraper](https://apify.com/automly/google-ads-transparency-scraper) | A competitor's Google ad creatives, regions and dates | ### ❓FAQ #### Do I need a Semrush account or API key? No. The scraper collects the figures Semrush shows to anyone without signing in, so you don't need a Semrush account, subscription or API key. Data Semrush keeps for paying subscribers (full keyword lists, individual backlinks, keyword research for any phrase) is not included. #### How many websites can I check in one run? As many as you like: paste hundreds or thousands of domains at once. #### Why is a website missing from my results? Semrush has no data for it without a subscription. Check the run report: the website is listed under `notFound`, and you are not charged for it. #### Why are some fields empty? Semrush does not show every figure for every website: smaller websites often have no competitors, no keywords or no country rank, and those fields are empty (never zero). #### Can I get keyword volume for any keyword? Not yet: Semrush shows keyword research only to signed-in users. Every website row does include its top organic keywords with search volume, CPC, position and intent. # Actor input Schema ## `domains` (type: `array`): Websites to look up, e.g. nike.com or https://www.stripe.com/. One per line; each gives one row with its Authority Score, traffic, backlinks, top keywords and competitors. Pages and subdomains are read as their main website. ## `topWebsitesCountries` (type: `array`): Countries to list the top websites of, e.g. United States or Worldwide. Leave empty to skip top websites. ## `topWebsitesIndustries` (type: `array`): Industries to list for each country, e.g. All industries or Finance. Each country and industry pair gives up to 100 websites ranked by monthly visits. ## `proxyConfiguration` (type: `object`): Apify Proxy works out of the box. ## Actor input object example ```json { "domains": [ "nike.com", "stripe.com" ], "topWebsitesCountries": [], "topWebsitesIndustries": [ "all" ], "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `domains` (type: `string`): No description ## `topWebsites` (type: `string`): No description ## `results` (type: `string`): No description ## `report` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "domains": [ "nike.com", "stripe.com" ], "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("automly/semrush-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "domains": [ "nike.com", "stripe.com", ], "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("automly/semrush-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "domains": [ "nike.com", "stripe.com" ], "proxyConfiguration": { "useApifyProxy": true } }' | apify call automly/semrush-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,automly/semrush-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/lHvhEvDu4bTzoyj06/builds/k1Kw9h9VfvZ7iFnbW/openapi.json