Bulk Domain Authority & Referring Domains - Open Data
Pricing
from $0.70 / 1,000 domain profiles
Bulk Domain Authority & Referring Domains - Open Data
Open Authority 0-100 and referring domains for any list of domains, plus referrer lists, link gap, similar sites and rank history to 2018. From the Common Crawl web graph (133M domains), Majestic and Chrome UX Report. No blocked runs; works as an MCP tool for AI agents.
Pricing
from $0.70 / 1,000 domain profiles
Rating
0.0
(0)
Developer
Piotr Zimniak
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
an hour ago
Last modified
Categories
Share
Independent tool, not affiliated with Common Crawl, Majestic, Google, Tranco, Ahrefs, Moz or Semrush. It reads only public open datasets and never visits the domains you check.
Authority, referring domains and real-traffic rank for any list of domains, from the open web graph instead of scraped
SEO tools. Every domain gets an Open Authority score from 0 to 100, computed from its rank among the 133 million domains
and 2.1 billion links of the Common Crawl web graph, its referring-domain count and how strong those referring domains
are. Other modes give the domains that link to it (referrers), the sites that link to your competitors but not to
you (linkGap), sites like it found from who links to them (similar), the domains it links to (outlinks)
and referring domains newly seen or no longer seen since the previous monthly graph (changes). Next to the score you
get Majestic Million referring subnets and IPs and the Chrome UX Report traffic bucket, a popularity rank based on
real Chrome visits. Paste 3 domains or 10,000.
Rank history goes back to 2018.
Why this one
- No captchas, no scraping. The data comes from public datasets, not from Ahrefs, Moz or Semrush pages. Nothing to solve, nothing to retry, no "try again later".
- Who links to you.
referrersmode lists the referring domains of each domain in your list, strongest first, each with its own Open Authority: 3,556 for apify.com, 249,224 for bbc.co.uk (September 2026 graph). - Link gap, from full lists.
linkGapcompares the referring domains of up to 20 competitors with yours and returns the ones you're missing, the domains linking to the most competitors first. Up to 100,000 of each competitor's strongest referring domains are compared, so the list is as deep as you need. - Find competitors you didn't know.
similarfinds the sites that the same websites link to: usually the same niche. Feed them straight intolinkGap. - Trend at a glance.
openAuthorityChange3mandopenAuthorityChange12mshow whether a domain is gaining or losing authority, from our monthly history of the graph. - How good the links are, not just how many.
ccReferringAuthoritySpreadsplits a domain's referring domains into Open Authority bands (90-100 ... 0-9), so a profile of a few strong sites and one of thousands of weak ones look different. - Links and real traffic in one row. You see the link-graph rank (Common Crawl), links from distinct networks (Majestic) and a Chrome-traffic bucket (CrUX) side by side. That shows sites that have links but no visitors, and sites that have visitors but few links. In a real run over 1,000 domains sampled across the Tranco top million, 399 were in the Majestic Million and 459 in the Chrome top million (September 2026).
- Fast in bulk. Ranks and referring-domain counts come from an index of the whole graph: every domain gets an exact rank, however small, and 1,000 domains take about a second on our side.
- History back to 2018. History mode returns the domain's rank in every Common Crawl web graph release you ask for, with a top-percent figure that stays comparable when the graph grows or shrinks.
- A score you can check. Open Authority is one published formula on a public rank (below).
- Handles any input format. URLs, hostnames, e-mail addresses and international domain names are all reduced to the registrable
domain (
https://www.bbc.co.uk/news→bbc.co.uk,info@scrapy.org→scrapy.org). - See it live. Where well-known domains sit on the web, the biggest 12-month climbers and a referring-domain profile, from this month's graph: crawlplant.com/commoncrawl-domain-metrics.
What can you use it for?
- Link building: rank a prospect list by authority and drop the long tail with
minAuthority; get an outreach list of strong sites that link to your competitors and not to you (linkGap). - Lead qualification: check whether a company's website has real traffic and links before sales spends time on it.
- SEO audits and competitor comparisons: authority, link popularity and traffic bucket for your site and its rivals.
- Research: popularity of domain lists, web graph studies (harmonic centrality, PageRank), longitudinal rank data.
- AI agents and RAG: weight or filter sources by domain authority before citing them.
Quick start
- Click Try for free (or Start) with the default input: three well-known domains, done in about 10 seconds, cost under $0.01 on the Free plan.
- Replace Domains with your list (one per line; URLs and e-mail addresses work).
- Download the table as CSV, Excel or JSON, or save the input as a task to rerun it.
Copy to your AI assistant
Paste this into ChatGPT, Claude or any agent so it knows how to use the Actor:
crawlplant/commoncrawl-domain-metrics on Apify: bulk domain authority from open data (no scraping of SEO tools).Input: domains (list of domains, URLs or e-mails; reduced to the registrable domain), mode ("profiles" default: one row perdomain; "history": one row per domain per Common Crawl web graph release, back to 2018; "referrers": one row per domainlinking to each of your domains, strongest first, up to maxReferrersPerDomain (default 100); "linkGap": domains linking tothe competitors in `domains` but not to `yourDomain`, most competitors first, comparing each competitor's strongestmaxReferrersPerDomain referring domains, hubs left out unless excludeHubs is false; "similar": sites like each domain by shared linking sites; "outlinks": domainseach domain links to; "changes": referring domains gained and lost since the previous monthly graph), sources (default["commoncrawl","majestic","crux"]; "tranco" adds the Tranco rank and, in history mode, its ~30-day daily history),graphScanDepth (only when the ranks file is read directly: history mode or an older graphRelease), minAuthority (0-100 filter;in referrers mode it filters the referring domains),sortBy ("input" | "authority"), historyReleases (default 6), graphRelease (release id, empty = newest), maxItems.Profile row: domain, openAuthority (0-100, from the Common Crawl harmonic-centrality rank r among N domains:round(100*(1-(ln r/ln N)^2))), openAuthorityMax (upper bound when the domain is below the searched depth), authorityTier(top-100 ... top-100m, long-tail, below-scan-depth, not-in-graph), ccHarmonicRank, ccHarmonicCentrality, ccPageRankRank,ccPageRank, ccTopPercent, ccHosts, ccReferringDomains (distinct domains linking to it in the graph),ccReferringAuthoritySpread (referring domains per Open Authority band, "90-100" ... "0-9"), openAuthorityChange3m andopenAuthorityChange12m (score now minus 3 / 12 monthly releases ago), ccRelease, majesticRank, majesticRefSubnets, majesticRefIps (+ previous day), cruxRankBucket(1000 = among the 1,000 most visited sites in Chrome ... 1000000), cruxOrigin, trancoRank. History row: domain, dataset(commoncrawl | tranco), date, rank, topPercent, openAuthority. Referrer row: domain, referringDomain, referringDomainAuthority,referringDomainRank, position (1 = strongest), referringDomains (total), ccRelease. Link-gap row: domain (yours),referringDomain, referringDomainAuthority, competitorsLinked, linksTo (competitor domains), ccRelease. Similar row: domain,similarDomain, similarity, sharedReferrers, similarDomainAuthority. Outlink row: domain, linkedDomain, linkedDomainAuthority,position. Change row: domain, change (gained | lost), referringDomain, referringDomainAuthority, gainedTotal, lostTotal,stableTotal, previousRelease. Default run ~5 s.
Modes
| Mode | What you get |
|---|---|
profiles (default) | One row per domain: Open Authority, Common Crawl ranks, Majestic links, Chrome traffic bucket (and Tranco if chosen) |
history | One row per domain per Common Crawl web graph release (newest first, back to 2018), plus Tranco daily ranks when tranco is a source |
referrers | One row per domain linking to each of your domains, strongest first, with its Open Authority |
linkGap | One row per domain that links to your competitors but not to you, the ones linking to the most competitors first |
similar | One row per site like each of your domains, most similar first (sites the same websites link to) |
outlinks | One row per domain each of your domains links to, strongest first |
changes | One row per referring domain gained or lost since the previous monthly graph |
Ready-to-use examples
1. Check domain authority for a list of domains
{ "domains": ["apify.com", "scrapy.org", "commoncrawl.org", "bbc.co.uk"] }
2. Qualify a lead list of e-mail addresses and website URLs
{ "domains": ["info@scrapy.org", "https://www.bbc.co.uk/news", "sales@apify.com"] }
3. Keep only link-building prospects with Open Authority 40 or more, strongest first
{ "domains": ["apify.com", "scrapy.org", "commoncrawl.org", "example.com"], "minAuthority": 40, "sortBy": "authority" }
4. Who links to a site: its 100 strongest referring domains
{ "mode": "referrers", "domains": ["apify.com"] }
5. Referring domains of competitors, only those with Open Authority 60 or more
{ "mode": "referrers", "domains": ["ahrefs.com", "semrush.com"], "minAuthority": 60, "maxReferrersPerDomain": 1000 }
6. Link popularity only: Majestic referring subnets and IPs
{ "domains": ["apify.com", "scrapy.org"], "sources": ["majestic"] }
7. Does this site get real visitors? Chrome traffic bucket only
{ "domains": ["apify.com", "scrapy.org"], "sources": ["crux"] }
8. Every source, including the Tranco research ranking
{ "domains": ["apify.com", "github.com"], "sources": ["commoncrawl", "majestic", "crux", "tranco"] }
9. Authority history over the last 12 graph releases
{ "mode": "history", "domains": ["apify.com", "scrapy.org"], "historyReleases": 12 }
10. Authority history since 2018
{ "mode": "history", "domains": ["bbc.co.uk"], "historyReleases": 60 }
11. Daily Tranco rank for the last month
{ "mode": "history", "domains": ["apify.com"], "sources": ["tranco"] }
12. Profiles from an older graph release (read from the ranks file, searched to graphScanDepth)
{ "domains": ["apify.com", "scrapy.org"], "graphRelease": "cc-main-2026-jun-jul-aug" }
13. Compare competitors, sorted by authority
{ "domains": ["ahrefs.com", "semrush.com", "moz.com", "majestic.com", "similarweb.com"], "sortBy": "authority" }
14. Link gap: sites that link to your competitors and not to you
{ "mode": "linkGap", "yourDomain": "scrapy.org", "domains": ["apify.com", "zyte.com", "scrapingbee.com"] }
15. Link gap, strong sites only, deeper comparison
{ "mode": "linkGap", "yourDomain": "scrapy.org", "domains": ["apify.com", "zyte.com"], "minAuthority": 50, "maxReferrersPerDomain": 20000 }
16. Find sites like yours (competitors and alternatives)
{ "mode": "similar", "domains": ["apify.com"], "maxReferrersPerDomain": 30 }
17. Which domains does a site link to?
{ "mode": "outlinks", "domains": ["scrapy.org"], "maxReferrersPerDomain": 200 }
18. Referring domains gained and lost since last month
{ "mode": "changes", "domains": ["apify.com", "scrapy.org"], "minAuthority": 30 }
How to…
Check domain authority in bulk
Paste the list into domains (example 1). Every domain in the Common Crawl graph gets its exact rank, score and
referring-domain count; a domain the graph doesn't have gets ccStatus: "not-in-graph" and a score of 0.
Find the sites that link to a domain
Use mode: "referrers" (examples 4 and 5). You get one row per referring domain, strongest first, with its Open Authority
and link rank. maxReferrersPerDomain sets how many per domain (default 100) and minAuthority keeps only the strong ones.
A referring domain is a domain with at least one link to yours in the pages of Common Crawl's last three monthly crawls.
Filter link-building prospects by authority
Set minAuthority (for example 40) and sortBy: "authority" (example 3). Domains under the minimum are left out, and you don't
pay for them. minAuthority: 1 drops only the domains without a score.
Find link-building opportunities (link gap)
Use mode: "linkGap" with your site in yourDomain and up to 20 competitors in domains (examples 14 and 15). Each row is
a domain that links to at least one competitor and not to you; competitorsLinked and linksTo say which. Rows come
sorted by the number of competitors, then by authority, so the top of the table is the best outreach list. Hubs are left
out by default (excludeHubs): infrastructure such as CDNs and link shorteners, and platforms that link to more than 30,000
domains; they link to everyone and are rarely a realistic pitch. Each
competitor's strongest maxReferrersPerDomain referring domains are compared (default 100; raise it for a deeper list,
up to 100,000), and minAuthority keeps only strong sites.
Find sites like a domain (competitor discovery)
Use mode: "similar" (example 16). Two sites that the same websites link to are usually in the same niche: the Actor
takes the strongest sites linking to your domain, looks at what else they link to and ranks the domains they share by
similarity. sharedReferrers says how many of the compared linking sites (referrersCompared) link to both. Big
general sites (search engines, social networks) are weighted down and infrastructure is left out. The result is a ready
competitor list for linkGap. It works best for companies and niche sites; for the biggest platforms (facebook.com,
google.com) almost every site links to them, so their similar sites are generic.
See which domains a site links to
Use mode: "outlinks" (example 17): the domains a site links to, strongest first, with their Open Authority.
See referring domains gained and lost since last month
Use mode: "changes" (example 18): one row per referring domain that appears in the newest monthly graph and not in the
previous one (gained), or the other way round (lost), strongest first, with the domain's totals (gainedTotal,
lostTotal, stableTotal). Each monthly graph is built from a different sample of pages, so a typical list turns over by
15-20% from one month to the next without any link being added or removed: read a change as "seen / no longer seen in
this month's crawl". The strong domains at the top of the lists and the totals over several months are the useful
signal; minAuthority keeps only those. Lists of up to 1,000,000 referring domains are compared; a bigger site (facebook.com has
17 million) gets a note in the run log instead of rows, and nothing is charged for it.
Judge the quality of a link profile
ccReferringAuthoritySpread counts a domain's referring domains per Open Authority band. Two sites with 5,000 referring
domains each can differ a lot: one with hundreds in the 60-100 bands has links from established sites, one with nearly
all of them in 0-9 has mostly a long tail of small or unlinked sites.
Check if a website gets real traffic
cruxRankBucket comes from the Chrome UX Report, based on real Chrome users. 1000 means the site is among the 1,000 most
visited sites in Chrome worldwide, then 5000, 10000, 50000, 100000, 500000 and 1000000. Empty means it is not in the top
million. Compare it with the link metrics: many links but no traffic bucket often means a link network, not a real audience.
Track a domain's authority over time
Profiles carry openAuthorityChange3m and openAuthorityChange12m: the score now minus 3 and 12 monthly releases ago.
For the full series use mode: "history" (examples 9 and 10): about a year of releases comes from our index for every
domain in seconds, and older releases back to 2018 are read from Common Crawl's files. A new web graph comes out about
every month, each covering the last three monthly crawls. Compare topPercent across releases rather than rank: the
number of domains in the graph changes between releases (from 70 to 209 million since 2018).
Weight RAG sources by domain authority
Run the domains of your retrieved sources and use openAuthority or authorityTier as a trust weight. The rows come back in
seconds, so the Actor works inside an agent loop (see Use with AI agents).
Get the Common Crawl harmonic centrality and PageRank of a domain
ccHarmonicRank / ccHarmonicCentrality and ccPageRankRank / ccPageRank are Common Crawl's own values for the domain in
the chosen release, the same numbers as in its published ranks file, with no extra processing.
Input options
| Option | Default | Description |
|---|---|---|
mode | profiles | profiles, history, referrers, linkGap, similar, outlinks or changes |
domains | 3 examples | Domains, URLs or e-mail addresses; reduced to the registrable domain, duplicates merged. linkGap: your competitors (up to 20) |
yourDomain | - | linkGap: your site; its referring domains are left out |
excludeHubs | true | linkGap: leave out infrastructure (CDNs, hosts of user content, link shorteners) and sites linking to more than 30,000 domains |
sources | commoncrawl, majestic, crux | Add tranco for the Tranco rank (and daily history in history mode) |
graphScanDepth | 10000000 | History mode and older releases: how many of the best-ranked domains to search in the ranks file; 0 = all 133M |
graphRelease | newest | A release id such as cc-main-2026-jun-jul-aug (list) |
minAuthority | 0 | Profiles: keep domains scoring at least this; 1 drops domains without a score. Link modes: keep the listed domains scoring at least this |
maxReferrersPerDomain | 100 | Rows per domain in referrers, similar, outlinks and changes (changes: per list), strongest first. linkGap: strongest referring domains compared per competitor (up to 100,000) |
sortBy | input | input or authority (highest first) |
historyReleases | 6 | History: how many graph releases, newest first |
maxItems | 10000 | Maximum rows |
Example output
A real row from September 2026:
{"type": "profile","domain": "apify.com","input": "apify.com","openAuthority": 79,"openAuthorityMax": null,"authorityTier": "top-10k","ccStatus": "found","ccHarmonicRank": 5287,"ccHarmonicCentrality": 17206752,"ccPageRankRank": 19402,"ccPageRank": 0.00000163412,"ccTopPercent": 0.003968,"ccHosts": 29,"ccReferringDomains": 3556,"ccScannedTo": null,"ccGraphDomains": 133241980,"ccRelease": "cc-main-2026-jul-aug-sep","ccReleaseDate": "2026-09-21","majesticRank": 16349,"majesticTldRank": 7895,"majesticRefSubnets": 2582,"majesticRefIps": 5805,"majesticPrevRank": 16436,"majesticPrevRefSubnets": 2571,"majesticPrevRefIps": 5754,"cruxRankBucket": 100000,"cruxOrigin": "https://apify.com","scrapedAt": "2026-09-29T16:51:35.073Z","source": "live"}
Output fields
| Field | Description |
|---|---|
type | profile or history |
domain, input | Registrable domain looked up (punycode for international names) and what you typed |
openAuthority | 0-100 score from the Common Crawl link-graph rank; null when the domain is below the searched depth |
openAuthorityMax | For domains below the searched depth: the highest score they could have |
authorityTier | top-100, top-1k, top-10k, top-100k, top-1m, top-10m, top-100m, long-tail, no-inlinks, below-scan-depth, not-in-graph, unavailable |
ccStatus | found, below-scan-depth (not in the searched top N), not-in-graph (whole graph searched), unavailable |
ccHarmonicRank, ccHarmonicCentrality | Rank and value of the domain's harmonic centrality: how close the rest of the web is to it through links |
ccPageRankRank, ccPageRank | Rank and value of its PageRank in the same graph |
ccTopPercent | Rank as a share of all domains in the graph (0.004 = top 0.004%) |
ccHosts | Hostnames of the domain seen in the graph |
ccReferringDomains | Distinct domains linking to it in the graph release |
ccReferringAuthoritySpread | Its referring domains per Open Authority band: 90-100, 80-89, ... 0-9 |
openAuthorityChange3m, openAuthorityChange12m | Open Authority now minus 3 / 12 monthly graph releases ago; null when the domain wasn't in that graph |
ccScannedTo | For domains below the depth: how deep the search went |
ccGraphDomains, ccRelease, ccReleaseDate | Size, id and approximate date (last crawl week) of the graph release |
majesticRank, majesticTldRank | Position in the Majestic Million, overall and within its top-level domain |
majesticRefSubnets, majesticRefIps | Distinct IP subnets (class C) and IP addresses of the sites linking to it |
majesticPrevRank, majesticPrevRefSubnets, majesticPrevRefIps | The same the day before |
cruxRankBucket, cruxOrigin | Chrome UX Report popularity bucket and the domain's best-ranked origin |
trancoRank, trancoListId, trancoListDate | With tranco: rank in the latest daily Tranco list (top 1M) |
dataset, period, date, rank, topPercent, graphDomains | History rows: source, release id or day, date, rank and top percent |
referringDomain, referringDomainAuthority, referringDomainRank, position, referringDomains | Referrer rows: the linking domain, its Open Authority and link rank, its place in the list (1 = strongest) and the total count |
competitorsLinked, linksTo, competitorsCompared | Link-gap rows: how many and which of your competitors the domain links to, out of how many compared (domain is yours) |
similarDomain, similarity, sharedReferrers, referrersCompared, similarDomainAuthority | Similar rows: the similar site, its similarity score, how many of the compared linking sites link to both, and its Open Authority |
linkedDomain, linkedDomainAuthority, linkedDomainRank, linkedDomains | Outlink rows: a domain it links to, its score and rank, and how many domains it links to in total |
change, stillInGraph, gainedTotal, lostTotal, stableTotal, previousRelease | Change rows: gained or lost since previousRelease, whether a lost domain is still in the graph, and the domain's totals |
scrapedAt, source | When the run read the data; always live |
Every field is null when the domain is not in that source's list (Majestic, CrUX and Tranco list the top million).
How is Open Authority calculated?
openAuthority = round(100 × (1 − (ln r / ln N)²)), where r is the domain's harmonic-centrality rank in the Common Crawl
web graph and N the number of domains in it. A domain nobody links to scores 0.
| Rank in the graph | Open Authority |
|---|---|
| 1 | 100 |
| 100 | 94 |
| 5,000 | 79 |
| 100,000 | 62 |
| 1,000,000 | 45 |
| 10,000,000 | 26 |
| 50,000,000 | 10 |
Harmonic centrality measures how close the rest of the web is to a domain through links, so a link from a well-connected site counts for more than many links from isolated ones. Common Crawl sorts its published domain ranks by it. Beyond the top ~20 million domains PageRank values are nearly all equal, which is why the score uses the harmonic rank.
Run it through the API
JavaScript:
import { ApifyClient } from 'apify-client';const client = new ApifyClient({ token: 'YOUR_TOKEN' });const run = await client.actor('crawlplant/commoncrawl-domain-metrics').call({ domains: ['apify.com', 'scrapy.org'], sortBy: 'authority' });const { items } = await client.dataset(run.defaultDatasetId).listItems();console.log(items.map((r) => `${r.domain}: ${r.openAuthority}`));
Python:
from apify_client import ApifyClientclient = ApifyClient("YOUR_TOKEN")run = client.actor("crawlplant/commoncrawl-domain-metrics").call(run_input={"domains": ["apify.com", "bbc.co.uk"]})for r in client.dataset(run["defaultDatasetId"]).iterate_items():print(r["domain"], r["openAuthority"], r["majesticRefSubnets"], r["cruxRankBucket"])
Use with AI agents
The Actor works as a tool in the Apify MCP server, for Claude, ChatGPT, Cursor, VS Code, LangChain and others. Every field is described. Server URL with only this tool:
https://mcp.apify.com?tools=crawlplant/commoncrawl-domain-metrics
Try "how authoritative are these five domains?", "which of my leads have websites with real traffic?" or "has example.com gained or lost authority since last year?" or "which strong sites link to my competitor?".
Pricing
Pay per result: platform usage is included. Each row type has one price, and you pay only for the rows you get: domains
left out by minAuthority are free.
| Event | No discount (Free plan) | Bronze (Starter) | Silver (Scale) | Gold (Business) |
|---|---|---|---|---|
| Domain profile (per 1,000) | $1.00 | $0.90 | $0.80 | $0.70 |
| Rank history row (per 1,000) | $1.50 | $1.35 | $1.20 | $1.05 |
| Link row: referring domain, outlink or change (per 1,000) | $2.00 | $1.80 | $1.60 | $1.40 |
| Link gap prospect (per 1,000) | $4.00 | $3.60 | $3.20 | $2.80 |
| Similar site (per 1,000) | $5.00 | $4.50 | $4.00 | $3.50 |
| Actor start (per run) | $0.00005 | $0.00005 | $0.00005 | $0.00005 |
| Example on the Free plan | Rows | Cost |
|---|---|---|
| Default run: 3 well-known domains | 3 | ~$0.003 |
| 1,000 domains across the web | 1,000 | ~$1.00 |
| 100 strongest referring domains of one site | 100 | ~$0.20 |
| Link gap, 500 outreach prospects | 500 | ~$2.00 |
| 50 sites similar to one domain | 50 | ~$0.25 |
| History: 10 domains × 6 releases | 60 | ~$0.09 |
Set a maximum cost per run in the run options to stop a large run at your budget.
Reliability
- There's nothing to block: the link data comes from our index of the Common Crawl web graph, rebuilt when a new release comes out, and the other sources are public dataset files (Majestic, GitHub, Tranco). No captchas, no logins, no browser.
- If the index can't be reached, profiles read Common Crawl's ranks file directly (down to
graphScanDepth, without referring-domain counts) and the run summary says so. - Each file is streamed from the top and the download stops as soon as every domain on your list is found. A dropped connection resumes where it stopped.
- If one source can't be read, the run still finishes with the other sources, and the run summary (
OUTPUTin the key-value store) says which source was missing and why. - The run respects its time limit: a very deep search stops in time and returns what it found, with a note.
Data sources and freshness
| Source | What it measures | Coverage | Updated |
|---|---|---|---|
| Common Crawl web graph | Links between 133M domains (harmonic centrality, PageRank, referring domains) | every linked domain | a new release about monthly, each covering 3 crawls; archive back to 2018 |
| Majestic Million | Referring subnets and IPs | top 1M | daily |
| Chrome UX Report top lists | Real Chrome visits, as a rank bucket | top 1M sites | monthly |
| Tranco (optional) | Research ranking combining several popularity lists | top 1M | daily; the API keeps about 30 days per domain |
Every run reads the newest data available at that moment. ccRelease, ccReleaseDate and trancoListDate say which
version you got.
Troubleshooting
below-scan-depthfor a domain I know: only when the ranks file is read directly (history mode, an oldergraphRelease): the domain ranks below the searched part (default: top 10 million). RaisegraphScanDepth, or set it to0.- A subdomain shows its parent domain: the graph ranks registrable domains, so
blog.example.comisexample.com. Hosting platforms count as one domain too (name.github.ioisgithub.io), as they do in Common Crawl. - Majestic, CrUX or Tranco fields are empty: those lists cover the top million sites only. An empty field means the domain isn't in that list, not an error.
- An input was skipped: IP addresses and text that isn't a domain are listed in the run summary.
- History has fewer releases than asked: a domain has rows only for releases where it ranked within
graphScanDepth, and history searches older releases only for domains found in the newest one.
FAQ
What is a good domain authority score?
On Open Authority, 45 is the top 1 million domains of 133 million, 62 the top 100,000 and 79 the top 5,000. Half of all domains in the graph score 7 or less.
Is Open Authority the same as Domain Rating or Domain Authority?
No. It is our own score from Common Crawl's public link graph, not Ahrefs DR or Moz DA, and the numbers are not meant to match. All of them grow with links from well-linked sites. Open Authority is free of any vendor's crawler and can be reproduced from public data.
How much does it cost to check 10,000 domains?
About $10 on the Free plan ($1.00 per 1,000 domain profiles). Link rows cost $2.00 per 1,000 (referring domains, outlinks, changes), link gap prospects $4.00 and similar sites $5.00 per 1,000. See Pricing.
Can it count backlinks or referring domains?
Yes, at the domain level: ccReferringDomains counts the distinct domains linking to each domain, and referrers mode lists
them, strongest first. The count comes from the pages of Common Crawl's last three monthly crawls, so it compares domains
fairly within one release; commercial backlink indexes crawl more pages and report higher totals. Majestic's referring
subnets and IPs are there too for the top million domains. It doesn't list individual backlink URLs or anchor texts.
How fresh is the data?
Each run reads the newest files: the Majestic Million from today, the newest Common Crawl web graph (about monthly) and the monthly Chrome UX Report list.
Does it visit or crawl my domains?
No. It only reads public datasets, so the sites you check never see a request.
Is it legal?
The Actor reads public datasets under their terms: Common Crawl's terms of use, Majestic Million under CC BY 3.0 and the Chrome UX Report under Creative Commons Attribution. The optional Tranco source combines several lists, including Cloudflare Radar (CC BY-NC 4.0). The sources are credited below. Check the terms that apply to your own use of the results.
Limits
- Referring domains are counted per registrable domain, from the newest graph release only (not in history mode).
- Majestic, CrUX and Tranco cover the top million domains or sites each.
- Tranco daily history is read for up to 50 domains per run (its API allows one request per second).
- One row per registrable domain: subdomains are merged into their domain.
Sources and credits
- Common Crawl web graph, commoncrawl.org, used under the Common Crawl terms of use.
- The Majestic Million by Majestic, licensed under CC BY 3.0.
- Chrome UX Report data by Google, via zakird/crux-top-lists.
- Tranco (optional source, off by default): V. Le Pochat et al., "Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation", NDSS 2019, tranco-list.eu. Its default list combines Chrome UX Report, Cloudflare Radar (CC BY-NC 4.0), Farsight, Majestic and Cisco Umbrella rankings.
Privacy
Only public datasets about domains are read; no personal data is collected. Each run sends the developer anonymous feature-usage statistics (the mode and which options were used, never the domains); your Apify account id is replaced by a one-way hash on arrival.