Bulk Domain Authority & Referring Domains - Open Data avatar

Bulk Domain Authority & Referring Domains - Open Data

Pricing

from $0.70 / 1,000 domain profiles

Go to Apify Store
Bulk Domain Authority & Referring Domains - Open Data

Bulk Domain Authority & Referring Domains - Open Data

Open Authority 0-100 and referring domains for any list of domains, plus referrer lists, link gap, similar sites and rank history to 2018. From the Common Crawl web graph (133M domains), Majestic and Chrome UX Report. No blocked runs; works as an MCP tool for AI agents.

Pricing

from $0.70 / 1,000 domain profiles

Rating

0.0

(0)

Developer

Piotr Zimniak

Piotr Zimniak

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

an hour ago

Last modified

Share

Independent tool, not affiliated with Common Crawl, Majestic, Google, Tranco, Ahrefs, Moz or Semrush. It reads only public open datasets and never visits the domains you check.

Authority, referring domains and real-traffic rank for any list of domains, from the open web graph instead of scraped SEO tools. Every domain gets an Open Authority score from 0 to 100, computed from its rank among the 133 million domains and 2.1 billion links of the Common Crawl web graph, its referring-domain count and how strong those referring domains are. Other modes give the domains that link to it (referrers), the sites that link to your competitors but not to you (linkGap), sites like it found from who links to them (similar), the domains it links to (outlinks) and referring domains newly seen or no longer seen since the previous monthly graph (changes). Next to the score you get Majestic Million referring subnets and IPs and the Chrome UX Report traffic bucket, a popularity rank based on real Chrome visits. Paste 3 domains or 10,000. Rank history goes back to 2018.

Why this one

  • No captchas, no scraping. The data comes from public datasets, not from Ahrefs, Moz or Semrush pages. Nothing to solve, nothing to retry, no "try again later".
  • Who links to you. referrers mode lists the referring domains of each domain in your list, strongest first, each with its own Open Authority: 3,556 for apify.com, 249,224 for bbc.co.uk (September 2026 graph).
  • Link gap, from full lists. linkGap compares the referring domains of up to 20 competitors with yours and returns the ones you're missing, the domains linking to the most competitors first. Up to 100,000 of each competitor's strongest referring domains are compared, so the list is as deep as you need.
  • Find competitors you didn't know. similar finds the sites that the same websites link to: usually the same niche. Feed them straight into linkGap.
  • Trend at a glance. openAuthorityChange3m and openAuthorityChange12m show whether a domain is gaining or losing authority, from our monthly history of the graph.
  • How good the links are, not just how many. ccReferringAuthoritySpread splits a domain's referring domains into Open Authority bands (90-100 ... 0-9), so a profile of a few strong sites and one of thousands of weak ones look different.
  • Links and real traffic in one row. You see the link-graph rank (Common Crawl), links from distinct networks (Majestic) and a Chrome-traffic bucket (CrUX) side by side. That shows sites that have links but no visitors, and sites that have visitors but few links. In a real run over 1,000 domains sampled across the Tranco top million, 399 were in the Majestic Million and 459 in the Chrome top million (September 2026).
  • Fast in bulk. Ranks and referring-domain counts come from an index of the whole graph: every domain gets an exact rank, however small, and 1,000 domains take about a second on our side.
  • History back to 2018. History mode returns the domain's rank in every Common Crawl web graph release you ask for, with a top-percent figure that stays comparable when the graph grows or shrinks.
  • A score you can check. Open Authority is one published formula on a public rank (below).
  • Handles any input format. URLs, hostnames, e-mail addresses and international domain names are all reduced to the registrable domain (https://www.bbc.co.uk/news → bbc.co.uk, info@scrapy.org → scrapy.org).
  • See it live. Where well-known domains sit on the web, the biggest 12-month climbers and a referring-domain profile, from this month's graph: crawlplant.com/commoncrawl-domain-metrics.

What can you use it for?

  • Link building: rank a prospect list by authority and drop the long tail with minAuthority; get an outreach list of strong sites that link to your competitors and not to you (linkGap).
  • Lead qualification: check whether a company's website has real traffic and links before sales spends time on it.
  • SEO audits and competitor comparisons: authority, link popularity and traffic bucket for your site and its rivals.
  • Research: popularity of domain lists, web graph studies (harmonic centrality, PageRank), longitudinal rank data.
  • AI agents and RAG: weight or filter sources by domain authority before citing them.

Quick start

  1. Click Try for free (or Start) with the default input: three well-known domains, done in about 10 seconds, cost under $0.01 on the Free plan.
  2. Replace Domains with your list (one per line; URLs and e-mail addresses work).
  3. Download the table as CSV, Excel or JSON, or save the input as a task to rerun it.

Copy to your AI assistant

Paste this into ChatGPT, Claude or any agent so it knows how to use the Actor:

crawlplant/commoncrawl-domain-metrics on Apify: bulk domain authority from open data (no scraping of SEO tools).
Input: domains (list of domains, URLs or e-mails; reduced to the registrable domain), mode ("profiles" default: one row per
domain; "history": one row per domain per Common Crawl web graph release, back to 2018; "referrers": one row per domain
linking to each of your domains, strongest first, up to maxReferrersPerDomain (default 100); "linkGap": domains linking to
the competitors in `domains` but not to `yourDomain`, most competitors first, comparing each competitor's strongest
maxReferrersPerDomain referring domains, hubs left out unless excludeHubs is false; "similar": sites like each domain by shared linking sites; "outlinks": domains
each domain links to; "changes": referring domains gained and lost since the previous monthly graph), sources (default
["commoncrawl","majestic","crux"]; "tranco" adds the Tranco rank and, in history mode, its ~30-day daily history),
graphScanDepth (only when the ranks file is read directly: history mode or an older graphRelease), minAuthority (0-100 filter;
in referrers mode it filters the referring domains),
sortBy ("input" | "authority"), historyReleases (default 6), graphRelease (release id, empty = newest), maxItems.
Profile row: domain, openAuthority (0-100, from the Common Crawl harmonic-centrality rank r among N domains:
round(100*(1-(ln r/ln N)^2))), openAuthorityMax (upper bound when the domain is below the searched depth), authorityTier
(top-100 ... top-100m, long-tail, below-scan-depth, not-in-graph), ccHarmonicRank, ccHarmonicCentrality, ccPageRankRank,
ccPageRank, ccTopPercent, ccHosts, ccReferringDomains (distinct domains linking to it in the graph),
ccReferringAuthoritySpread (referring domains per Open Authority band, "90-100" ... "0-9"), openAuthorityChange3m and
openAuthorityChange12m (score now minus 3 / 12 monthly releases ago), ccRelease, majesticRank, majesticRefSubnets, majesticRefIps (+ previous day), cruxRankBucket
(1000 = among the 1,000 most visited sites in Chrome ... 1000000), cruxOrigin, trancoRank. History row: domain, dataset
(commoncrawl | tranco), date, rank, topPercent, openAuthority. Referrer row: domain, referringDomain, referringDomainAuthority,
referringDomainRank, position (1 = strongest), referringDomains (total), ccRelease. Link-gap row: domain (yours),
referringDomain, referringDomainAuthority, competitorsLinked, linksTo (competitor domains), ccRelease. Similar row: domain,
similarDomain, similarity, sharedReferrers, similarDomainAuthority. Outlink row: domain, linkedDomain, linkedDomainAuthority,
position. Change row: domain, change (gained | lost), referringDomain, referringDomainAuthority, gainedTotal, lostTotal,
stableTotal, previousRelease. Default run ~5 s.

Modes

ModeWhat you get
profiles (default)One row per domain: Open Authority, Common Crawl ranks, Majestic links, Chrome traffic bucket (and Tranco if chosen)
historyOne row per domain per Common Crawl web graph release (newest first, back to 2018), plus Tranco daily ranks when tranco is a source
referrersOne row per domain linking to each of your domains, strongest first, with its Open Authority
linkGapOne row per domain that links to your competitors but not to you, the ones linking to the most competitors first
similarOne row per site like each of your domains, most similar first (sites the same websites link to)
outlinksOne row per domain each of your domains links to, strongest first
changesOne row per referring domain gained or lost since the previous monthly graph

Ready-to-use examples

1. Check domain authority for a list of domains

{ "domains": ["apify.com", "scrapy.org", "commoncrawl.org", "bbc.co.uk"] }

2. Qualify a lead list of e-mail addresses and website URLs

{ "domains": ["info@scrapy.org", "https://www.bbc.co.uk/news", "sales@apify.com"] }

3. Keep only link-building prospects with Open Authority 40 or more, strongest first

{ "domains": ["apify.com", "scrapy.org", "commoncrawl.org", "example.com"], "minAuthority": 40, "sortBy": "authority" }

4. Who links to a site: its 100 strongest referring domains

{ "mode": "referrers", "domains": ["apify.com"] }

5. Referring domains of competitors, only those with Open Authority 60 or more

{ "mode": "referrers", "domains": ["ahrefs.com", "semrush.com"], "minAuthority": 60, "maxReferrersPerDomain": 1000 }

6. Link popularity only: Majestic referring subnets and IPs

{ "domains": ["apify.com", "scrapy.org"], "sources": ["majestic"] }

7. Does this site get real visitors? Chrome traffic bucket only

{ "domains": ["apify.com", "scrapy.org"], "sources": ["crux"] }

8. Every source, including the Tranco research ranking

{ "domains": ["apify.com", "github.com"], "sources": ["commoncrawl", "majestic", "crux", "tranco"] }

9. Authority history over the last 12 graph releases

{ "mode": "history", "domains": ["apify.com", "scrapy.org"], "historyReleases": 12 }

10. Authority history since 2018

{ "mode": "history", "domains": ["bbc.co.uk"], "historyReleases": 60 }

11. Daily Tranco rank for the last month

{ "mode": "history", "domains": ["apify.com"], "sources": ["tranco"] }

12. Profiles from an older graph release (read from the ranks file, searched to graphScanDepth)

{ "domains": ["apify.com", "scrapy.org"], "graphRelease": "cc-main-2026-jun-jul-aug" }

13. Compare competitors, sorted by authority

{ "domains": ["ahrefs.com", "semrush.com", "moz.com", "majestic.com", "similarweb.com"], "sortBy": "authority" }

14. Link gap: sites that link to your competitors and not to you

{ "mode": "linkGap", "yourDomain": "scrapy.org", "domains": ["apify.com", "zyte.com", "scrapingbee.com"] }

15. Link gap, strong sites only, deeper comparison

{ "mode": "linkGap", "yourDomain": "scrapy.org", "domains": ["apify.com", "zyte.com"], "minAuthority": 50, "maxReferrersPerDomain": 20000 }

16. Find sites like yours (competitors and alternatives)

{ "mode": "similar", "domains": ["apify.com"], "maxReferrersPerDomain": 30 }

17. Which domains does a site link to?

{ "mode": "outlinks", "domains": ["scrapy.org"], "maxReferrersPerDomain": 200 }

18. Referring domains gained and lost since last month

{ "mode": "changes", "domains": ["apify.com", "scrapy.org"], "minAuthority": 30 }

How to…

Check domain authority in bulk

Paste the list into domains (example 1). Every domain in the Common Crawl graph gets its exact rank, score and referring-domain count; a domain the graph doesn't have gets ccStatus: "not-in-graph" and a score of 0.

Use mode: "referrers" (examples 4 and 5). You get one row per referring domain, strongest first, with its Open Authority and link rank. maxReferrersPerDomain sets how many per domain (default 100) and minAuthority keeps only the strong ones. A referring domain is a domain with at least one link to yours in the pages of Common Crawl's last three monthly crawls.

Set minAuthority (for example 40) and sortBy: "authority" (example 3). Domains under the minimum are left out, and you don't pay for them. minAuthority: 1 drops only the domains without a score.

Use mode: "linkGap" with your site in yourDomain and up to 20 competitors in domains (examples 14 and 15). Each row is a domain that links to at least one competitor and not to you; competitorsLinked and linksTo say which. Rows come sorted by the number of competitors, then by authority, so the top of the table is the best outreach list. Hubs are left out by default (excludeHubs): infrastructure such as CDNs and link shorteners, and platforms that link to more than 30,000 domains; they link to everyone and are rarely a realistic pitch. Each competitor's strongest maxReferrersPerDomain referring domains are compared (default 100; raise it for a deeper list, up to 100,000), and minAuthority keeps only strong sites.

Find sites like a domain (competitor discovery)

Use mode: "similar" (example 16). Two sites that the same websites link to are usually in the same niche: the Actor takes the strongest sites linking to your domain, looks at what else they link to and ranks the domains they share by similarity. sharedReferrers says how many of the compared linking sites (referrersCompared) link to both. Big general sites (search engines, social networks) are weighted down and infrastructure is left out. The result is a ready competitor list for linkGap. It works best for companies and niche sites; for the biggest platforms (facebook.com, google.com) almost every site links to them, so their similar sites are generic.

Use mode: "outlinks" (example 17): the domains a site links to, strongest first, with their Open Authority.

See referring domains gained and lost since last month

Use mode: "changes" (example 18): one row per referring domain that appears in the newest monthly graph and not in the previous one (gained), or the other way round (lost), strongest first, with the domain's totals (gainedTotal, lostTotal, stableTotal). Each monthly graph is built from a different sample of pages, so a typical list turns over by 15-20% from one month to the next without any link being added or removed: read a change as "seen / no longer seen in this month's crawl". The strong domains at the top of the lists and the totals over several months are the useful signal; minAuthority keeps only those. Lists of up to 1,000,000 referring domains are compared; a bigger site (facebook.com has 17 million) gets a note in the run log instead of rows, and nothing is charged for it.

ccReferringAuthoritySpread counts a domain's referring domains per Open Authority band. Two sites with 5,000 referring domains each can differ a lot: one with hundreds in the 60-100 bands has links from established sites, one with nearly all of them in 0-9 has mostly a long tail of small or unlinked sites.

Check if a website gets real traffic

cruxRankBucket comes from the Chrome UX Report, based on real Chrome users. 1000 means the site is among the 1,000 most visited sites in Chrome worldwide, then 5000, 10000, 50000, 100000, 500000 and 1000000. Empty means it is not in the top million. Compare it with the link metrics: many links but no traffic bucket often means a link network, not a real audience.

Track a domain's authority over time

Profiles carry openAuthorityChange3m and openAuthorityChange12m: the score now minus 3 and 12 monthly releases ago. For the full series use mode: "history" (examples 9 and 10): about a year of releases comes from our index for every domain in seconds, and older releases back to 2018 are read from Common Crawl's files. A new web graph comes out about every month, each covering the last three monthly crawls. Compare topPercent across releases rather than rank: the number of domains in the graph changes between releases (from 70 to 209 million since 2018).

Weight RAG sources by domain authority

Run the domains of your retrieved sources and use openAuthority or authorityTier as a trust weight. The rows come back in seconds, so the Actor works inside an agent loop (see Use with AI agents).

Get the Common Crawl harmonic centrality and PageRank of a domain

ccHarmonicRank / ccHarmonicCentrality and ccPageRankRank / ccPageRank are Common Crawl's own values for the domain in the chosen release, the same numbers as in its published ranks file, with no extra processing.

Input options

OptionDefaultDescription
modeprofilesprofiles, history, referrers, linkGap, similar, outlinks or changes
domains3 examplesDomains, URLs or e-mail addresses; reduced to the registrable domain, duplicates merged. linkGap: your competitors (up to 20)
yourDomain-linkGap: your site; its referring domains are left out
excludeHubstruelinkGap: leave out infrastructure (CDNs, hosts of user content, link shorteners) and sites linking to more than 30,000 domains
sourcescommoncrawl, majestic, cruxAdd tranco for the Tranco rank (and daily history in history mode)
graphScanDepth10000000History mode and older releases: how many of the best-ranked domains to search in the ranks file; 0 = all 133M
graphReleasenewestA release id such as cc-main-2026-jun-jul-aug (list)
minAuthority0Profiles: keep domains scoring at least this; 1 drops domains without a score. Link modes: keep the listed domains scoring at least this
maxReferrersPerDomain100Rows per domain in referrers, similar, outlinks and changes (changes: per list), strongest first. linkGap: strongest referring domains compared per competitor (up to 100,000)
sortByinputinput or authority (highest first)
historyReleases6History: how many graph releases, newest first
maxItems10000Maximum rows

Example output

A real row from September 2026:

{
"type": "profile",
"domain": "apify.com",
"input": "apify.com",
"openAuthority": 79,
"openAuthorityMax": null,
"authorityTier": "top-10k",
"ccStatus": "found",
"ccHarmonicRank": 5287,
"ccHarmonicCentrality": 17206752,
"ccPageRankRank": 19402,
"ccPageRank": 0.00000163412,
"ccTopPercent": 0.003968,
"ccHosts": 29,
"ccReferringDomains": 3556,
"ccScannedTo": null,
"ccGraphDomains": 133241980,
"ccRelease": "cc-main-2026-jul-aug-sep",
"ccReleaseDate": "2026-09-21",
"majesticRank": 16349,
"majesticTldRank": 7895,
"majesticRefSubnets": 2582,
"majesticRefIps": 5805,
"majesticPrevRank": 16436,
"majesticPrevRefSubnets": 2571,
"majesticPrevRefIps": 5754,
"cruxRankBucket": 100000,
"cruxOrigin": "https://apify.com",
"scrapedAt": "2026-09-29T16:51:35.073Z",
"source": "live"
}

Output fields

FieldDescription
typeprofile or history
domain, inputRegistrable domain looked up (punycode for international names) and what you typed
openAuthority0-100 score from the Common Crawl link-graph rank; null when the domain is below the searched depth
openAuthorityMaxFor domains below the searched depth: the highest score they could have
authorityTiertop-100, top-1k, top-10k, top-100k, top-1m, top-10m, top-100m, long-tail, no-inlinks, below-scan-depth, not-in-graph, unavailable
ccStatusfound, below-scan-depth (not in the searched top N), not-in-graph (whole graph searched), unavailable
ccHarmonicRank, ccHarmonicCentralityRank and value of the domain's harmonic centrality: how close the rest of the web is to it through links
ccPageRankRank, ccPageRankRank and value of its PageRank in the same graph
ccTopPercentRank as a share of all domains in the graph (0.004 = top 0.004%)
ccHostsHostnames of the domain seen in the graph
ccReferringDomainsDistinct domains linking to it in the graph release
ccReferringAuthoritySpreadIts referring domains per Open Authority band: 90-100, 80-89, ... 0-9
openAuthorityChange3m, openAuthorityChange12mOpen Authority now minus 3 / 12 monthly graph releases ago; null when the domain wasn't in that graph
ccScannedToFor domains below the depth: how deep the search went
ccGraphDomains, ccRelease, ccReleaseDateSize, id and approximate date (last crawl week) of the graph release
majesticRank, majesticTldRankPosition in the Majestic Million, overall and within its top-level domain
majesticRefSubnets, majesticRefIpsDistinct IP subnets (class C) and IP addresses of the sites linking to it
majesticPrevRank, majesticPrevRefSubnets, majesticPrevRefIpsThe same the day before
cruxRankBucket, cruxOriginChrome UX Report popularity bucket and the domain's best-ranked origin
trancoRank, trancoListId, trancoListDateWith tranco: rank in the latest daily Tranco list (top 1M)
dataset, period, date, rank, topPercent, graphDomainsHistory rows: source, release id or day, date, rank and top percent
referringDomain, referringDomainAuthority, referringDomainRank, position, referringDomainsReferrer rows: the linking domain, its Open Authority and link rank, its place in the list (1 = strongest) and the total count
competitorsLinked, linksTo, competitorsComparedLink-gap rows: how many and which of your competitors the domain links to, out of how many compared (domain is yours)
similarDomain, similarity, sharedReferrers, referrersCompared, similarDomainAuthoritySimilar rows: the similar site, its similarity score, how many of the compared linking sites link to both, and its Open Authority
linkedDomain, linkedDomainAuthority, linkedDomainRank, linkedDomainsOutlink rows: a domain it links to, its score and rank, and how many domains it links to in total
change, stillInGraph, gainedTotal, lostTotal, stableTotal, previousReleaseChange rows: gained or lost since previousRelease, whether a lost domain is still in the graph, and the domain's totals
scrapedAt, sourceWhen the run read the data; always live

Every field is null when the domain is not in that source's list (Majestic, CrUX and Tranco list the top million).

How is Open Authority calculated?

openAuthority = round(100 × (1 − (ln r / ln N)²)), where r is the domain's harmonic-centrality rank in the Common Crawl web graph and N the number of domains in it. A domain nobody links to scores 0.

Rank in the graphOpen Authority
1100
10094
5,00079
100,00062
1,000,00045
10,000,00026
50,000,00010

Harmonic centrality measures how close the rest of the web is to a domain through links, so a link from a well-connected site counts for more than many links from isolated ones. Common Crawl sorts its published domain ranks by it. Beyond the top ~20 million domains PageRank values are nearly all equal, which is why the score uses the harmonic rank.

Run it through the API

JavaScript:

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('crawlplant/commoncrawl-domain-metrics').call({ domains: ['apify.com', 'scrapy.org'], sortBy: 'authority' });
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map((r) => `${r.domain}: ${r.openAuthority}`));

Python:

from apify_client import ApifyClient
client = ApifyClient("YOUR_TOKEN")
run = client.actor("crawlplant/commoncrawl-domain-metrics").call(run_input={"domains": ["apify.com", "bbc.co.uk"]})
for r in client.dataset(run["defaultDatasetId"]).iterate_items():
print(r["domain"], r["openAuthority"], r["majesticRefSubnets"], r["cruxRankBucket"])

Use with AI agents

The Actor works as a tool in the Apify MCP server, for Claude, ChatGPT, Cursor, VS Code, LangChain and others. Every field is described. Server URL with only this tool:

https://mcp.apify.com?tools=crawlplant/commoncrawl-domain-metrics

Try "how authoritative are these five domains?", "which of my leads have websites with real traffic?" or "has example.com gained or lost authority since last year?" or "which strong sites link to my competitor?".

Pricing

Pay per result: platform usage is included. Each row type has one price, and you pay only for the rows you get: domains left out by minAuthority are free.

EventNo discount (Free plan)Bronze (Starter)Silver (Scale)Gold (Business)
Domain profile (per 1,000)$1.00$0.90$0.80$0.70
Rank history row (per 1,000)$1.50$1.35$1.20$1.05
Link row: referring domain, outlink or change (per 1,000)$2.00$1.80$1.60$1.40
Link gap prospect (per 1,000)$4.00$3.60$3.20$2.80
Similar site (per 1,000)$5.00$4.50$4.00$3.50
Actor start (per run)$0.00005$0.00005$0.00005$0.00005
Example on the Free planRowsCost
Default run: 3 well-known domains3~$0.003
1,000 domains across the web1,000~$1.00
100 strongest referring domains of one site100~$0.20
Link gap, 500 outreach prospects500~$2.00
50 sites similar to one domain50~$0.25
History: 10 domains × 6 releases60~$0.09

Set a maximum cost per run in the run options to stop a large run at your budget.

Reliability

  • There's nothing to block: the link data comes from our index of the Common Crawl web graph, rebuilt when a new release comes out, and the other sources are public dataset files (Majestic, GitHub, Tranco). No captchas, no logins, no browser.
  • If the index can't be reached, profiles read Common Crawl's ranks file directly (down to graphScanDepth, without referring-domain counts) and the run summary says so.
  • Each file is streamed from the top and the download stops as soon as every domain on your list is found. A dropped connection resumes where it stopped.
  • If one source can't be read, the run still finishes with the other sources, and the run summary (OUTPUT in the key-value store) says which source was missing and why.
  • The run respects its time limit: a very deep search stops in time and returns what it found, with a note.

Data sources and freshness

SourceWhat it measuresCoverageUpdated
Common Crawl web graphLinks between 133M domains (harmonic centrality, PageRank, referring domains)every linked domaina new release about monthly, each covering 3 crawls; archive back to 2018
Majestic MillionReferring subnets and IPstop 1Mdaily
Chrome UX Report top listsReal Chrome visits, as a rank buckettop 1M sitesmonthly
Tranco (optional)Research ranking combining several popularity liststop 1Mdaily; the API keeps about 30 days per domain

Every run reads the newest data available at that moment. ccRelease, ccReleaseDate and trancoListDate say which version you got.

Troubleshooting

  • below-scan-depth for a domain I know: only when the ranks file is read directly (history mode, an older graphRelease): the domain ranks below the searched part (default: top 10 million). Raise graphScanDepth, or set it to 0.
  • A subdomain shows its parent domain: the graph ranks registrable domains, so blog.example.com is example.com. Hosting platforms count as one domain too (name.github.io is github.io), as they do in Common Crawl.
  • Majestic, CrUX or Tranco fields are empty: those lists cover the top million sites only. An empty field means the domain isn't in that list, not an error.
  • An input was skipped: IP addresses and text that isn't a domain are listed in the run summary.
  • History has fewer releases than asked: a domain has rows only for releases where it ranked within graphScanDepth, and history searches older releases only for domains found in the newest one.

FAQ

What is a good domain authority score?

On Open Authority, 45 is the top 1 million domains of 133 million, 62 the top 100,000 and 79 the top 5,000. Half of all domains in the graph score 7 or less.

Is Open Authority the same as Domain Rating or Domain Authority?

No. It is our own score from Common Crawl's public link graph, not Ahrefs DR or Moz DA, and the numbers are not meant to match. All of them grow with links from well-linked sites. Open Authority is free of any vendor's crawler and can be reproduced from public data.

How much does it cost to check 10,000 domains?

About $10 on the Free plan ($1.00 per 1,000 domain profiles). Link rows cost $2.00 per 1,000 (referring domains, outlinks, changes), link gap prospects $4.00 and similar sites $5.00 per 1,000. See Pricing.

Yes, at the domain level: ccReferringDomains counts the distinct domains linking to each domain, and referrers mode lists them, strongest first. The count comes from the pages of Common Crawl's last three monthly crawls, so it compares domains fairly within one release; commercial backlink indexes crawl more pages and report higher totals. Majestic's referring subnets and IPs are there too for the top million domains. It doesn't list individual backlink URLs or anchor texts.

How fresh is the data?

Each run reads the newest files: the Majestic Million from today, the newest Common Crawl web graph (about monthly) and the monthly Chrome UX Report list.

Does it visit or crawl my domains?

No. It only reads public datasets, so the sites you check never see a request.

The Actor reads public datasets under their terms: Common Crawl's terms of use, Majestic Million under CC BY 3.0 and the Chrome UX Report under Creative Commons Attribution. The optional Tranco source combines several lists, including Cloudflare Radar (CC BY-NC 4.0). The sources are credited below. Check the terms that apply to your own use of the results.

Limits

  • Referring domains are counted per registrable domain, from the newest graph release only (not in history mode).
  • Majestic, CrUX and Tranco cover the top million domains or sites each.
  • Tranco daily history is read for up to 50 domains per run (its API allows one request per second).
  • One row per registrable domain: subdomains are merged into their domain.

Sources and credits

  • Common Crawl web graph, commoncrawl.org, used under the Common Crawl terms of use.
  • The Majestic Million by Majestic, licensed under CC BY 3.0.
  • Chrome UX Report data by Google, via zakird/crux-top-lists.
  • Tranco (optional source, off by default): V. Le Pochat et al., "Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation", NDSS 2019, tranco-list.eu. Its default list combines Chrome UX Report, Cloudflare Radar (CC BY-NC 4.0), Farsight, Majestic and Cisco Umbrella rankings.

Privacy

Only public datasets about domains are read; no personal data is collected. Each run sends the developer anonymous feature-usage statistics (the mode and which options were used, never the domains); your Apify account id is replaced by a one-way hash on arrival.