Ahrefs Scraper: Domain Rating, Traffic, AI Leaderboards

Ahrefs scraper: domain metrics, 11 years of history and AI leaderboards

Three jobs in one actor. Pull the Ahrefs authority and traffic profile for any list of domains, the public ranking of the most visited sites, or the brands and sources that AI assistants cite.

$1 per 1,000 results, plus $0.005 per run.

Why this scraper?

• 📈 11 years of monthly history on every domain. Traffic, traffic value and referring domains, month by month, back to 2015. Ahrefs' own page charts only the last year.
• 🤖 AI leaderboards nobody else publishes. Which brands ChatGPT, Gemini, Perplexity and Copilot mention most, and which sites they cite, for the last 12 months.
• 🧮 Exact numbers, not rounded ones. Where the page shows "10.9M" this returns 10890577, and where it shows "3.1M" of US traffic it returns 3069964.
• 💸 You are never billed for a domain we could not find. Unindexed domains come back marked found: false and cost nothing.
• 🔎 Traffic even where the page shows none. Some sites display no traffic figure at all. This still returns one, recovered from the underlying monthly series.
• 🌍 Per country traffic split with last month's figure, the change, and share of total.
• 🏢 Competitors on request. Each domain's rivals with their own Domain Rating, traffic, traffic value and page count.
• 🔁 Bulk friendly. Paste a full URL, a bare domain, or the same site three different ways. Duplicates collapse to one row and one charge, and a typo is skipped rather than killing the run.
• 🚀 Fast. A twenty domain run finishes in about six seconds. Nothing to configure before you start.

What data can you extract?

🏆 Domain Rating	🔗 Linking websites	📊 Organic traffic	💵 Traffic value
🔑 Organic keywords	🌐 Global rank	🏷️ Category rank	📂 Category
📈 Traffic history	💰 Traffic value history	🔗 Referring domain history	🌍 Traffic by country
🏢 Competitors	↕️ Rank movement	🤖 Crawl permission	🕓 Snapshot timestamp
🥇 Top sites by country	📂 Top sites by category	🔥 Trending up and down	🧠 AI brand mentions

The history is the part nobody else gives you

Every domain row carries two full monthly series:

• history: organic traffic and estimated traffic value, per month
• refdomainsHistory: referring domain count, per month

For an established site that is about 135 monthly data points each, starting June 2015. Younger domains return their full life, so a site launched in 2020 comes back with roughly 70 points rather than a padded series.

Enough to plot a five year trend, spot the month a competitor's rankings collapsed, or check whether a domain you are about to buy has been declining quietly for two years.

Scraping modes

🔍 Domain metrics

Pass your own list of domains. One row each.

{
  "mode": "domain",
  "domains": ["shopify.com", "stripe.com", "notion.so"],
  "includeHistory": true,
  "includeCountryBreakdown": true,
  "includeCompetitors": false
}

🏆 Top websites

The public ranking of the most visited sites. Country and category combine, so Australia plus Finance gives you the top Australian finance sites.

{
  "mode": "top-websites",
  "country": "australia",
  "category": "finance",
  "listMode": "ranking",
  "maxItems": 100
}

Set listMode to trending for the 10 biggest climbers and the 10 biggest fallers, each row tagged up or down.

🧠 AI leaderboards

The brands AI assistants mention most, and the websites they cite most, month by month.

{
  "mode": "ai-leaderboards",
  "aiPlatforms": ["Chatgpt", "Gemini", "Perplexity"],
  "aiMonthsBack": 12,
  "aiLeaderboard": "both"
}

Pick any of Google AI Overviews, Google AI Mode, ChatGPT, Gemini, Perplexity and Copilot, or all for the combined board. Twelve months of history are published.

How to use it: step by step

1. Create a free Apify account. Takes 30 seconds, no card needed.
2. Open the Actor in the Apify Console.
3. Pick a mode. For domain metrics, paste your domains one per line: a full URL works, it is reduced to the domain for you.
4. Click Start. Most runs finish in seconds.
5. Export as JSON, CSV or Excel, or fetch it through the API.

Input

{
  "mode": "domain",
  "domains": ["shopify.com", "https://www.stripe.com/pricing", "notion.so"],
  "includeHistory": true,
  "includeCountryBreakdown": true,
  "includeRobotsStatus": false,
  "includeCompetitors": false,
  "maxItems": 100,
  "maxConcurrency": 10
}

Field	Type	Default	Notes
mode	string	domain	domain, top-websites or ai-leaderboards
domains	array	none	Domain metrics only. One per line, bare domain or full URL
includeHistory	boolean	true	Adds both monthly series. Costs no extra requests
includeCountryBreakdown	boolean	true	Adds the per country split
includeRobotsStatus	boolean	false	Adds crawl permission. One extra request per domain
includeCompetitors	boolean	false	Adds competing sites. One extra request per domain
country	string	worldwide	Top websites only. A slug such as united-states
category	string	all	Top websites only
listMode	string	ranking	Top websites only. ranking or trending
aiPlatforms	array	["all"]	AI leaderboards only
aiMonthsBack	integer	1	AI leaderboards only. Up to 12
aiLeaderboard	string	both	AI leaderboards only. both, brands or sources
maxItems	integer	1000	Ceiling on billed rows
maxConcurrency	integer	10	Requests in parallel. Higher finishes sooner and costs less

Output

Domain metrics, trimmed here for readability:

{
  "mode": "domain",
  "domain": "shopify.com",
  "found": true,
  "domainRating": 96,
  "linkingWebsites": 1182025,
  "linkingWebsitesDelta": 48478,
  "organicTraffic": 10890577,
  "organicTrafficDelta": 1997337,
  "trafficValueUsd": 12890180,
  "trafficValueDeltaUsd": 2426603,
  "organicKeywords": 1300000,
  "rankGlobal": 563,
  "rankGlobalDelta": 129,
  "rankCategory": 70,
  "category": "Shopping",
  "trafficByCountry": [
    { "country": "United States", "countryCode": "US", "organicTraffic": 3069964, "organicTrafficPrev": 2514606, "delta": 555358, "sharePct": 28.1 }
  ],
  "history": [
    { "month": "2015-06-01", "organicTraffic": 603421, "trafficValueUsd": 1135289 }
  ],
  "refdomainsHistory": [
    { "month": "2015-04-01", "refdomains": 78400 }
  ],
  "historyMonths": 135,
  "competitors": null,
  "scrapedAt": "2026-08-27T13:20:11.402Z"
}

An AI leaderboards row:

{
  "mode": "ai-leaderboards",
  "platform": "Chatgpt",
  "month": "2026-07-01",
  "leaderboard": "sources",
  "rank": 1,
  "label": "reddit.com",
  "url": "https://reddit.com",
  "value": 998569,
  "scrapedAt": "2026-08-27T13:20:11.402Z"
}

A domain that is not in the index comes back like this, and is not charged:

{
  "mode": "domain",
  "domain": "some-tiny-site.dev",
  "found": false,
  "error": "not indexed by the source"
}

rankGlobal is null for any site outside the published top 1,000. When that happens rankGlobalLabel holds what the source actually shows, usually ">1000", so you can tell "outside the leaderboard" apart from "we could not read it".

Use cases

• 📊 Competitor benchmarking. Line up twenty competitor domains and compare authority, traffic and traffic value side by side.
• 📉 Spot a decline before you buy. Pull five years of monthly traffic on a domain you are about to acquire.
• 🧠 AI search visibility. Track whether your brand is gaining or losing mentions across ChatGPT, Gemini and Perplexity month over month.
• 🧲 Lead scoring. Score inbound leads or prospect lists by Domain Rating and organic traffic.
• 🔗 Link prospecting triage. Rank outreach targets by authority and linking websites before you spend a single email.
• 🕵️ Market mapping. Pull the leaders in a country or category, then enrich each one.
• 📚 Portfolio monitoring. Watch a set of owned or client domains and catch a traffic drop the month it happens.

Limitations

• Domain coverage is the public Ahrefs index, roughly 185,000 sites. Large and established sites are there. Smaller and newer niche sites often are not, and return an unbilled found: false row.
• The per country split is the top 5 countries. That is what the source publishes, not a truncation on our side.
• linkingWebsites is not the same number as Ahrefs' refdomains. It is the figure the public page reports and it can differ from the number inside a paid Ahrefs account. It is named differently on purpose so you never mistake one for the other.
• The top websites ranking is 100 rows, trending is 20. Ask for more and the run says so plainly rather than pretending.
• AI leaderboards are 10 entries per board. Six AI platforms report separately; the rest of the market is only available combined.
• No keyword research, SERP data, or individual backlink lists. This actor answers how strong a domain is and how it is trending, not what it ranks for.
• Some domains carry no category or no rank. Where the source leaves them blank the field is null rather than guessed.

FAQ

How much does the Ahrefs scraper cost?

Pay per result. $1 for 1,000 results, the same in all three modes, plus $0.005 each time a run starts. Domains that are not in the index are never charged.

The Apify Free plan gives you $5 in usage credits a month, which is about 5,000 results. The $29/month Starter plan covers about 29,000 results. Both figures are before the half-cent per run, so it is worth batching domains into one run rather than starting fifty.

No subscription lock-in. Pause whenever.

Is it legal to scrape this data?

Scraping public data is generally allowed in the US and most of the EU, as long as you don't collect personal data covered by GDPR or CCPA without a lawful basis. This actor only touches publicly accessible pages, but how you use the output is on you.

Apify's full breakdown: Is web scraping legal?.

Can I integrate it with other tools?

Push results into Make, Zapier, Slack, Airbyte, GitHub, Google Sheets, Google Drive, and more. Apify treats every actor as a webhook source, so anything that consumes webhooks or pulls from an API works.

Full list: Apify integrations.

Can I use it with the Apify API?

Yes. Every run is available through the Apify REST API:

curl -X POST "https://api.apify.com/v2/acts/sourabhbgp~ahrefs-scraper/runs?token=APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"domain","domains":["shopify.com"]}'

Docs: Apify API reference.

Can I use it through an MCP Server?

Yes. Apify ships an MCP server that exposes every actor as a tool, so Claude Desktop, Cursor, and any other MCP-capable client can call it. Setup: Apify MCP docs.

Your feedback

Bug, missing field, or odd behavior? Drop a note in the Issues tab. Reports go to a human and fixes usually ship the same week.