# Ahrefs Scraper: Domain Rating, Traffic, AI Leaderboards (`sourabhbgp/ahrefs-scraper`) Actor Give it a list of domains and get Ahrefs Domain Rating, organic traffic, traffic value and 11 years of monthly history. Also pulls the public top websites ranking and the brands AI assistants mention most. - **URL**: https://apify.com/sourabhbgp/ahrefs-scraper.md - **Developed by:** [Sourabh Kumar](https://apify.com/sourabhbgp) (community) - **Categories:** SEO tools, Developer tools, AI - **Stats:** 1 total users, 0 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ### Ahrefs scraper: domain metrics, 11 years of history and AI leaderboards Three jobs in one actor. Pull the Ahrefs authority and traffic profile for any list of domains, the public ranking of the most visited sites, or the brands and sources that AI assistants cite. **$1 per 1,000 results, plus $0.005 per run.** ### Why this scraper? - 📈 **11 years of monthly history on every domain.** Traffic, traffic value and referring domains, month by month, back to 2015. Ahrefs' own page charts only the last year. - 🤖 **AI leaderboards nobody else publishes.** Which brands ChatGPT, Gemini, Perplexity and Copilot mention most, and which sites they cite, for the last 12 months. - 🧮 **Exact numbers, not rounded ones.** Where the page shows "10.9M" this returns `10890577`, and where it shows "3.1M" of US traffic it returns `3069964`. - 💸 **You are never billed for a domain we could not find.** Unindexed domains come back marked `found: false` and cost nothing. - 🔎 **Traffic even where the page shows none.** Some sites display no traffic figure at all. This still returns one, recovered from the underlying monthly series. - 🌍 **Per country traffic split** with last month's figure, the change, and share of total. - 🏢 **Competitors on request.** Each domain's rivals with their own Domain Rating, traffic, traffic value and page count. - 🔁 **Bulk friendly.** Paste a full URL, a bare domain, or the same site three different ways. Duplicates collapse to one row and one charge, and a typo is skipped rather than killing the run. - 🚀 **Fast.** A twenty domain run finishes in about six seconds. Nothing to configure before you start. ### What data can you extract?
🏆 Domain Rating🔗 Linking websites📊 Organic traffic💵 Traffic value
🔑 Organic keywords🌐 Global rank🏷️ Category rank📂 Category
📈 Traffic history💰 Traffic value history🔗 Referring domain history🌍 Traffic by country
🏢 Competitors↕️ Rank movement🤖 Crawl permission🕓 Snapshot timestamp
🥇 Top sites by country📂 Top sites by category🔥 Trending up and down🧠 AI brand mentions
### The history is the part nobody else gives you Every domain row carries two full monthly series: - `history`: organic traffic and estimated traffic value, per month - `refdomainsHistory`: referring domain count, per month For an established site that is about **135 monthly data points each**, starting June 2015. Younger domains return their full life, so a site launched in 2020 comes back with roughly 70 points rather than a padded series. Enough to plot a five year trend, spot the month a competitor's rankings collapsed, or check whether a domain you are about to buy has been declining quietly for two years. ### Scraping modes #### 🔍 Domain metrics Pass your own list of domains. One row each. ```json { "mode": "domain", "domains": ["shopify.com", "stripe.com", "notion.so"], "includeHistory": true, "includeCountryBreakdown": true, "includeCompetitors": false } ``` #### 🏆 Top websites The public ranking of the most visited sites. Country and category combine, so Australia plus Finance gives you the top Australian finance sites. ```json { "mode": "top-websites", "country": "australia", "category": "finance", "listMode": "ranking", "maxItems": 100 } ``` Set `listMode` to `trending` for the 10 biggest climbers and the 10 biggest fallers, each row tagged `up` or `down`. #### 🧠 AI leaderboards The brands AI assistants mention most, and the websites they cite most, month by month. ```json { "mode": "ai-leaderboards", "aiPlatforms": ["Chatgpt", "Gemini", "Perplexity"], "aiMonthsBack": 12, "aiLeaderboard": "both" } ``` Pick any of Google AI Overviews, Google AI Mode, ChatGPT, Gemini, Perplexity and Copilot, or `all` for the combined board. Twelve months of history are published. ### How to use it: step by step 1. [Create a free Apify account](https://console.apify.com/sign-up). Takes 30 seconds, no card needed. 2. Open the Actor in the [Apify Console](https://console.apify.com/actors/NGqSriIpoaonAxnUU). 3. Pick a mode. For domain metrics, paste your domains one per line: a full URL works, it is reduced to the domain for you. 4. Click **Start**. Most runs finish in seconds. 5. Export as JSON, CSV or Excel, or fetch it through the API. ### Input ```json { "mode": "domain", "domains": ["shopify.com", "https://www.stripe.com/pricing", "notion.so"], "includeHistory": true, "includeCountryBreakdown": true, "includeRobotsStatus": false, "includeCompetitors": false, "maxItems": 100, "maxConcurrency": 10 } ``` | Field | Type | Default | Notes | |---|---|---|---| | `mode` | string | `domain` | `domain`, `top-websites` or `ai-leaderboards` | | `domains` | array | none | Domain metrics only. One per line, bare domain or full URL | | `includeHistory` | boolean | `true` | Adds both monthly series. Costs no extra requests | | `includeCountryBreakdown` | boolean | `true` | Adds the per country split | | `includeRobotsStatus` | boolean | `false` | Adds crawl permission. One extra request per domain | | `includeCompetitors` | boolean | `false` | Adds competing sites. One extra request per domain | | `country` | string | `worldwide` | Top websites only. A slug such as `united-states` | | `category` | string | all | Top websites only | | `listMode` | string | `ranking` | Top websites only. `ranking` or `trending` | | `aiPlatforms` | array | `["all"]` | AI leaderboards only | | `aiMonthsBack` | integer | `1` | AI leaderboards only. Up to 12 | | `aiLeaderboard` | string | `both` | AI leaderboards only. `both`, `brands` or `sources` | | `maxItems` | integer | `1000` | Ceiling on billed rows | | `maxConcurrency` | integer | `10` | Requests in parallel. Higher finishes sooner and costs less | ### Output Domain metrics, trimmed here for readability: ```json { "mode": "domain", "domain": "shopify.com", "found": true, "domainRating": 96, "linkingWebsites": 1182025, "linkingWebsitesDelta": 48478, "organicTraffic": 10890577, "organicTrafficDelta": 1997337, "trafficValueUsd": 12890180, "trafficValueDeltaUsd": 2426603, "organicKeywords": 1300000, "rankGlobal": 563, "rankGlobalDelta": 129, "rankCategory": 70, "category": "Shopping", "trafficByCountry": [ { "country": "United States", "countryCode": "US", "organicTraffic": 3069964, "organicTrafficPrev": 2514606, "delta": 555358, "sharePct": 28.1 } ], "history": [ { "month": "2015-06-01", "organicTraffic": 603421, "trafficValueUsd": 1135289 } ], "refdomainsHistory": [ { "month": "2015-04-01", "refdomains": 78400 } ], "historyMonths": 135, "competitors": null, "scrapedAt": "2026-08-27T13:20:11.402Z" } ``` An AI leaderboards row: ```json { "mode": "ai-leaderboards", "platform": "Chatgpt", "month": "2026-07-01", "leaderboard": "sources", "rank": 1, "label": "reddit.com", "url": "https://reddit.com", "value": 998569, "scrapedAt": "2026-08-27T13:20:11.402Z" } ``` A domain that is not in the index comes back like this, and is not charged: ```json { "mode": "domain", "domain": "some-tiny-site.dev", "found": false, "error": "not indexed by the source" } ``` `rankGlobal` is `null` for any site outside the published top 1,000. When that happens `rankGlobalLabel` holds what the source actually shows, usually `">1000"`, so you can tell "outside the leaderboard" apart from "we could not read it". ### Use cases - 📊 **Competitor benchmarking.** Line up twenty competitor domains and compare authority, traffic and traffic value side by side. - 📉 **Spot a decline before you buy.** Pull five years of monthly traffic on a domain you are about to acquire. - 🧠 **AI search visibility.** Track whether your brand is gaining or losing mentions across ChatGPT, Gemini and Perplexity month over month. - 🧲 **Lead scoring.** Score inbound leads or prospect lists by Domain Rating and organic traffic. - 🔗 **Link prospecting triage.** Rank outreach targets by authority and linking websites before you spend a single email. - 🕵️ **Market mapping.** Pull the leaders in a country or category, then enrich each one. - 📚 **Portfolio monitoring.** Watch a set of owned or client domains and catch a traffic drop the month it happens. ### Limitations - **Domain coverage is the public Ahrefs index**, roughly 185,000 sites. Large and established sites are there. Smaller and newer niche sites often are not, and return an unbilled `found: false` row. - **The per country split is the top 5 countries.** That is what the source publishes, not a truncation on our side. - **`linkingWebsites` is not the same number as Ahrefs' `refdomains`.** It is the figure the public page reports and it can differ from the number inside a paid Ahrefs account. It is named differently on purpose so you never mistake one for the other. - **The top websites ranking is 100 rows, trending is 20.** Ask for more and the run says so plainly rather than pretending. - **AI leaderboards are 10 entries per board.** Six AI platforms report separately; the rest of the market is only available combined. - **No keyword research, SERP data, or individual backlink lists.** This actor answers how strong a domain is and how it is trending, not what it ranks for. - **Some domains carry no category or no rank.** Where the source leaves them blank the field is `null` rather than guessed. ### FAQ #### How much does the Ahrefs scraper cost? Pay per result. **$1 for 1,000 results**, the same in all three modes, plus **$0.005 each time a run starts**. Domains that are not in the index are never charged. The Apify Free plan gives you $5 in usage credits a month, which is about **5,000 results**. The $29/month Starter plan covers about **29,000 results**. Both figures are before the half-cent per run, so it is worth batching domains into one run rather than starting fifty. No subscription lock-in. Pause whenever. #### Is it legal to scrape this data? Scraping public data is generally allowed in the US and most of the EU, as long as you don't collect personal data covered by GDPR or CCPA without a lawful basis. This actor only touches publicly accessible pages, but how you use the output is on you. Apify's full breakdown: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/). #### Can I integrate it with other tools? Push results into **Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive**, and more. Apify treats every actor as a webhook source, so anything that consumes webhooks or pulls from an API works. Full list: [Apify integrations](https://docs.apify.com/platform/integrations). #### Can I use it with the Apify API? Yes. Every run is available through the Apify REST API: ```bash curl -X POST "https://api.apify.com/v2/acts/sourabhbgp~ahrefs-scraper/runs?token=APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"mode":"domain","domains":["shopify.com"]}' ``` Docs: [Apify API reference](https://docs.apify.com/api/v2). #### Can I use it through an MCP Server? Yes. Apify ships an MCP server that exposes every actor as a tool, so Claude Desktop, Cursor, and any other MCP-capable client can call it. Setup: [Apify MCP docs](https://docs.apify.com/platform/integrations/mcp). ### Your feedback Bug, missing field, or odd behavior? Drop a note in the [Issues tab](https://console.apify.com/actors/NGqSriIpoaonAxnUU/issues). Reports go to a human and fixes usually ship the same week. # Actor input Schema ## `mode` (type: `string`): Domain metrics: pass your own list of domains and get one row each. Top websites: pull the public ranking of the most visited sites. AI leaderboards: the brands AI assistants mention most and the websites they cite most, by month. ## `domains` (type: `array`): One domain per line. A full URL is fine: it is reduced to the domain automatically. Entries that are not domains are skipped and reported, they do not stop the run. Used by the Domain metrics mode only. ## `includeHistory` (type: `boolean`): Adds the full monthly series of organic traffic, traffic value and referring domains, going back as far as the site is indexed (up to about 11 years). Costs no extra requests. ## `includeCountryBreakdown` (type: `boolean`): Adds the per country split of organic traffic with month over month change and share. The source publishes the top 5 countries. ## `includeRobotsStatus` (type: `boolean`): Adds whether AhrefsBot is allowed to crawl the domain, per host and per port. Costs one extra request per domain. ## `includeCompetitors` (type: `boolean`): Adds the competing sites the source lists for each domain, with their Domain Rating, traffic, traffic value, pages and a short description. Costs one extra request per domain. ## `country` (type: `string`): Top websites mode only. Use worldwide, or a country slug such as united-states or germany. An unknown slug ends the run cleanly and charges nothing. ## `category` (type: `string`): Top websites mode only. Optional category filter. Combines with country, so Australia plus Finance returns the top Australian finance sites. ## `listMode` (type: `string`): Top websites mode only. Ranking returns the top 100 sites. Trending returns the 10 biggest climbers and 10 biggest fallers, each tagged with a direction. ## `aiPlatforms` (type: `array`): AI leaderboards mode only. Pick one or more AI platforms, or All platforms for the combined board. Only platforms the source actually reports separately are listed here. ## `aiMonthsBack` (type: `integer`): AI leaderboards mode only. How many months back from the most recent published month. The source publishes 12 months. ## `aiLeaderboard` (type: `string`): AI leaderboards mode only. Brands are the names AI assistants mention. Sources are the websites they cite. ## `maxItems` (type: `integer`): Hard ceiling on billed rows, for every mode. Rows that are not charged (a domain the source does not index, an entry that is not a domain) do not count against it. ## `maxConcurrency` (type: `integer`): How many requests to run in parallel. Higher is both faster AND cheaper: compute is billed on memory multiplied by run time, so finishing sooner costs less than the extra parallelism does. ## `proxyConfiguration` (type: `object`): Defaults to Apify's automatic routing, which uses datacenter IPs. The Actor escalates to residential by itself if datacenter is refused, so there is normally nothing to change here. ## Actor input object example ```json { "mode": "domain", "domains": [ "shopify.com", "stripe.com", "notion.so" ], "includeHistory": true, "includeCountryBreakdown": true, "includeRobotsStatus": false, "includeCompetitors": false, "country": "worldwide", "category": "", "listMode": "ranking", "aiPlatforms": [ "all" ], "aiMonthsBack": 1, "aiLeaderboard": "both", "maxItems": 100, "maxConcurrency": 10, "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `datasetItems` (type: `string`): Every scraped row. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "mode": "domain", "domains": [ "shopify.com", "stripe.com", "notion.so" ], "includeHistory": true, "includeCountryBreakdown": true, "aiPlatforms": [ "all" ], "aiMonthsBack": 1, "maxItems": 100, "maxConcurrency": 10, "proxyConfiguration": { "useApifyProxy": true } }; // Run the Actor and wait for it to finish const run = await client.actor("sourabhbgp/ahrefs-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "mode": "domain", "domains": [ "shopify.com", "stripe.com", "notion.so", ], "includeHistory": True, "includeCountryBreakdown": True, "aiPlatforms": ["all"], "aiMonthsBack": 1, "maxItems": 100, "maxConcurrency": 10, "proxyConfiguration": { "useApifyProxy": True }, } # Run the Actor and wait for it to finish run = client.actor("sourabhbgp/ahrefs-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "mode": "domain", "domains": [ "shopify.com", "stripe.com", "notion.so" ], "includeHistory": true, "includeCountryBreakdown": true, "aiPlatforms": [ "all" ], "aiMonthsBack": 1, "maxItems": 100, "maxConcurrency": 10, "proxyConfiguration": { "useApifyProxy": true } }' | apify call sourabhbgp/ahrefs-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,sourabhbgp/ahrefs-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/NGqSriIpoaonAxnUU/builds/7DK6lGA01mAyBMY8Y/openapi.json