# Domain Overview - Semrush & Ahrefs Alternative, Tech & Hiring (`tidytools/domain-overview`) Actor

One row per domain: traffic rank (global and by country, 12-month trend), authority and referring domains, tech stack, open jobs, domain age and AI crawler policy. Bulk lists, no logins. $10 per 1,000 domains; unreadable websites free.

- **URL**: https://apify.com/tidytools/domain-overview.md
- **Developed by:** [Yukai Lin](https://apify.com/tidytools) (community)
- **Categories:** SEO tools, Lead generation, Marketing
- **Stats:** 3 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $8.00 / 1,000 domain analyzeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Domain Overview do?

**Input:** a list of domains (or URLs, e-mail addresses, or another run's dataset). **Output:** one row per domain with traffic rank, authority, tech stack, open jobs, domain age and AI crawler policy. **Price:** $10 per 1,000 domains ($0.01 each), only when the website could be read; unreadable websites, duplicates and invalid lines are free. No start fee.

**Honest scope:** this is not Semrush or Ahrefs data. Traffic is a Chrome UX Report rank tier (Top 1K ... Top 1M, per country), not a visit count; authority is our own 0-100 score from the Common Crawl web graph, not Ahrefs DR; there are no organic keywords or backlink lists. What you get instead is open data plus the live website, live job openings and DNS/RDAP records, in one table.

For each domain you get **one row** that answers "how big is this company's website, what is it built with, and is the company growing?":

- 📈 **Traffic rank** from real Chrome users: global Top 1K ... 1M, the top country, the rank in up to 238 countries and the 6-month trend
- 🔗 **Authority**: a 0-100 score and the number of referring domains (who links to the site), from the Common Crawl web graph
- 🧰 **Tech stack**: CMS, e-commerce platform, frameworks, analytics, CDN and hosting, plus the **e-mail provider** and the SaaS tools verified in its DNS
- 💼 **Hiring**: open jobs, remote jobs, hiring countries, sample job titles and the careers page, from our index of 1.4 million openings on 54,000 company career sites (Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Personio and 10 more)
- 🗓️ **Domain age** and registrar (from the registry's RDAP record)
- 🤖 **AI crawler policy**: whether robots.txt lets GPTBot, ClaudeBot, PerplexityBot and 26 other AI crawlers in, and whether the site has an `llms.txt`
- 📝 the site's **title and description**, and a one-line **summary**

Pick only the sections you need; the price is the same.

#### Real results (3 October 2026)

| Domain | Summary (as returned) |
|---|---|
| stripe.com | Top 10K worldwide (stable); authority 85/100 (710,778 referring domains); Next.js, React, LiveRamp, Meta Business; 711 open jobs; domain 31 years old; open to AI crawlers |
| figma.com | Top 5K worldwide (stable); authority 73/100 (44,879 referring domains); Sanity, Shopify, Next.js, React; 164 open jobs; domain 27.4 years old; AI training crawlers blocked, AI search allowed |
| airbnb.com | Top 5K worldwide (stable); authority 72/100 (76,816 referring domains); Segment, Meta Business, Yahoo, Envoy; 150 open jobs; domain 18.1 years old; AI training crawlers blocked, AI search allowed |
| notion.com | Top 5K worldwide (rising); authority 67/100 (19,140 referring domains); Contentful, Next.js, React, Tailwind CSS; 134 open jobs; domain 28.9 years old; some AI search crawlers blocked |
| linear.app | Top 50K worldwide (stable); authority 61/100 (4,288 referring domains); Next.js, React, Cloudflare, Google Cloud; 30 open jobs; domain 8.4 years old; open to AI crawlers |
| apify.com | Top 100K worldwide (rising); authority 60/100 (3,556 referring domains); Next.js, React, Bootstrap, Mixpanel; 11 open jobs; domain 17.3 years old; open to AI crawlers |
| bbc.co.uk | Top 1K worldwide (stable); authority 81/100 (249,224 referring domains); Comscore, Yahoo, Fastly, Varnish; domain 31.8 years old; some AI search crawlers blocked |
| wordpress.org | Top 50K worldwide (stable); authority 91/100 (2,806,838 referring domains); WordPress 7.2, Google Tag Manager, Nginx, PHP; domain 23.5 years old; open to AI crawlers |

### Use cases

- **Lead scoring and account research**: turn a list of company websites into size (traffic, authority), stack (what they use), growth (open jobs) and age, in one table, before you reach out.
- **Sales triggers**: "uses Shopify, Top 100K in the US, hiring" is a filter, not a research project.
- **SEO and partnership prospecting**: authority and traffic for link-building, guest-post or affiliate lists, without an Ahrefs or Semrush seat.
- **Market maps and due diligence**: compare every company in a niche on the same metrics.
- **GEO / AI visibility checks**: which sites block AI search crawlers and which publish `llms.txt`.
- **Enrich another run**: pass the dataset of a Google Maps, directory or lead scraper and its `website` field.

### How much does it cost?

| Event | Price | When |
|---|---|---|
| Domain analyzed | **$10.00 / 1,000 domains** ($0.01 each) | the domain's website was read, with every section you chose |

**Free:** websites that cannot be read (down, does not exist, parked or for sale, "coming soon" or example pages, blocked, or robots.txt disallows our crawler). Their row still carries the traffic, authority, hiring and domain-age data we have. Duplicates (`stripe.com` and `https://www.stripe.com/pricing` are one domain) and invalid lines are free too. **No start fee.** Larger Apify plans get the usual volume discount (Silver 10%, Gold and above 20%).

**Example:** 500 company websites from a Google Maps run cost at most **$5**.

**Speed:** each website is read live, so speed depends on the sites: a list of 500 mixed domains (about 30 real sites, the rest not resolving) took about 6 minutes with the default 8 parallel reads, and 10 real sites take about 10 seconds. Raise **Websites in parallel** (up to 20) for long lists; a slow site is cut off after the time limit and is free.

For comparison (Apify Store, 3 October 2026): Actors that return Ahrefs or Semrush domain metrics charge $0.01-$0.02 per domain (the most used: $0.015 per row, 1,056 and 573 users in 30 days); Similarweb Actors $0.002-$0.02; tech stack lookups $0.002-$0.1. Those return the commercial tools' own numbers (Ahrefs DR, organic keywords, estimated visits), which this Actor does not have. This Actor combines open data, the live website and live job openings, and adds hiring, domain age and AI crawler policy that those tools do not return.

#### Control your cost

- **Before anything is read**, the run logs its plan (domains x price = the most the run can cost) and compares it with your **maximum charge per run**. When the next domain could pass it, no new domain is started: `SUMMARY.status` is `LIMIT_REACHED` and `SUMMARY.notProcessed` lists the domains not processed.
- Every row has `charged: true` or `false`; free rows explain why in `error` and `websiteStatus`.
- If Apify restarts the run, domains already done are skipped and not charged again.

### How to use it

1. Paste domains, URLs or e-mail addresses into **Domains**, one per line, or pick a **Dataset ID** (and **Dataset field**).
2. Optional: choose **Sections** (empty = all).
3. Click **Start**. **Domains (overview)** shows the summary and the key numbers; **Domains (all fields)** has everything, ready for CSV or Excel export.

Traffic, authority and hiring are looked up for the registrable domain (`shop.example.co.uk` -> `example.co.uk`); the website read is the host you gave. Entries can be separated by line breaks, spaces, commas or semicolons; surrounding quotes, brackets and `<...>` from pasted lists are ignored. `www.`, ports, paths, e-mail addresses and international names (`bücher.de`) are accepted. If you give several hosts of one domain (`blog.example.com`, `www.example.com`), the domain is analyzed once, from the first host, and charged once; all spellings are listed in `inputs`.

#### Input example

This is the input the form is prefilled with; a run with it takes about a minute and costs only cents:

```json
{
    "domains": [
        "stripe.com",
        "figma.com",
        "apify.com",
        "bbc.co.uk",
        "tools.yukai.uk"
    ]
}
```

#### Output example

Real result (3 October 2026), shortened:

```json
{
    "input": "linear.app",
    "domain": "linear.app",
    "website": "https://linear.app/",
    "websiteStatus": "online",
    "title": "Linear – The system for product development",
    "globalTrafficRank": 50000,
    "globalTrafficRankLabel": "Top 50K websites worldwide",
    "trafficTrend": "stable",
    "topCountry": "us",
    "topCountryRankLabel": "Top 5K in United States",
    "countriesListed": 127,
    "authorityScore": 61,
    "referringDomains": 4288,
    "technologyCount": 24,
    "technologies": ["Next.js", "React", "Cloudflare", "Google Cloud", "Stripe", "HubSpot", "Google Workspace", "Slack", "Zoom"],
    "hosting": "Cloudflare, Google Cloud",
    "emailProvider": "Google Workspace",
    "aiCrawlerPolicy": "open",
    "aiBotsBlocked": [],
    "hasLlmsTxt": true,
    "domainCreatedAt": "2018-05-09T23:45:22.384Z",
    "domainAgeYears": 8.4,
    "registrar": "CloudFlare, Inc.",
    "hiringIndexed": true,
    "openJobs": 30,
    "remoteJobs": 29,
    "hiringCountries": ["US", "European Union", "GB"],
    "sampleJobTitles": ["Senior / Staff Fullstack Engineer", "Product Manager", "Developer Relations"],
    "careersUrl": "https://jobs.ashbyhq.com/linear",
    "atsPlatforms": ["ashby"],
    "summary": "Top 50K worldwide (stable); authority 61/100 (4,288 referring domains); Next.js, React, Cloudflare, Google Cloud; 30 open jobs; domain 8.4 years old; open to AI crawlers",
    "charged": true
}
```

#### What the fields mean

- **Traffic**: Chrome UX Report rank buckets (Google, monthly, CC BY 4.0). `globalTrafficRank: 50000` = among the 50,000 most visited origins worldwide. `countriesListed: 0` = not in any country's list (very little traffic). Details: Website Traffic Rank Checker by TidyTools.
- **Authority**: our transparent 0-100 score from the Common Crawl web graph (July-September 2026) and the count of referring domains. Not Ahrefs DR or Moz DA. Details: Domain Authority Checker by TidyTools.
- **Hiring**: `hiringIndexed: false` means we found no careers board for the company on the 16 job platforms we index, so its hiring is **unknown** (`openJobs: null`), not zero (a company whose board has no live openings at the moment looks the same).
- **Tech stack**: detected from the home page (HTML, headers, cookies, scripts) and public DNS records (MX, SPF, verification TXT records), the same engine as the Tech Stack Detector by TidyTools.
- **AI crawler policy**: `open`, `search-only` (only AI training crawlers blocked), `blocks-search` (some AI search or assistant crawlers blocked), `restrictive` (all of them blocked), `no-robots` or `unknown`.

### Use it from code

```bash
curl -X POST "https://api.apify.com/v2/acts/tidytools~domain-overview/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains":["stripe.com","linear.app"],"sections":["traffic","techStack","hiring"]}'
```

**With AI agents (MCP):** connect Apify's MCP server (`https://mcp.apify.com?tools=tidytools/domain-overview`) to Claude, Cursor or any MCP client and ask, for example, "Give me an overview of these 20 competitors: size, stack and who is hiring". Minimal input: `{"domains": ["stripe.com"]}`.

### Limitations

- **No keyword, organic search or visit numbers**: traffic is a Chrome rank tier per country, not visits; there are no organic keywords or backlink lists. Use an SEO suite if you need those.
- **Hiring covers companies with a careers page on one of the 16 job platforms we index** (54,000 companies). Companies that post only on LinkedIn, Indeed or their own custom site show `hiringIndexed: false`.
- The tech stack is read from the home page only (one page per domain); technologies used only after login or on other pages are not seen.
- Domain age needs a public RDAP record; some country domains (for example .de, whose registry does not publish registration dates) and registries without RDAP return `domainAgeYears: null` with `domainAgeError`.
- Each request names us (TidyTools user agent). If robots.txt disallows our crawler on the home page, the website is not read (free row).

### Responsible use

The Actor reads public company websites, public DNS and RDAP records, public job postings and open datasets. It does not collect personal data: no names, personal e-mail addresses or phone numbers. Registrant details in RDAP records are not returned. Use the results in line with the websites' terms and the laws that apply to you.

### FAQ

**Does the Actor collect usage data?** The Actor sends one anonymous usage counter per run (input type, item count range, outcome) to improve the tool; no URLs, content or account data.

**Is this the same data as Semrush or Ahrefs?** No. It is open data (Chrome UX Report, Common Crawl) plus what the website and job boards publish. It is consistent across millions of domains and cheap enough for whole lead lists, but the numbers will not match Semrush or Ahrefs.

**What happens when part of the data is missing?** Each section stands alone. A domain that is not in the traffic lists says so (`countriesListed: 0`, "low traffic"); one that no other domain links to has `authorityScore: 0`; one with no careers board in our index has `hiringIndexed: false` and `openJobs: null` (unknown, not zero); a registry without an RDAP record gives `domainAgeYears: null` and `domainAgeError`. If one of our data lookups itself fails, that section is `null` and `SUMMARY.dataLookupFailures` counts it; the rest of the row is still returned.

**What happens with a wrong or empty input?** Lines that are not a domain (an IP address, "localhost", a bare "co.uk", a name with an unknown ending such as `.invalidtld` or `.local`, an invalid name such as `-bad-.com`, text) get a row with `errorType: "invalid_input"` and `charged: false`; if nothing valid is left the run still ends normally (SUMMARY.status `FAILED`, nothing charged). A Dataset ID that cannot be read gives one free error row, and any domains you pasted are still analyzed.

**Why was a domain not charged?** Its website could not be read (see `websiteStatus` and `error`). The data sections we have are still in the row.

### Support

Open an issue in the **Issues** tab with the domains and the run link. Issues are checked regularly.

# Actor input Schema

## `domains` (type: `array`):

Domains, website addresses or e-mail addresses, one per line (stripe.com, https://www.example.com/page, info@example.com). Data is looked up for the registrable domain (shop.example.co.uk -> example.co.uk); the website read is the host you give. Spaces, commas and semicolons also separate entries. Up to 10,000 per run; duplicates are merged. Invalid lines become a result row with errorType "invalid_input" (not charged).

## `datasetId` (type: `string`):

Analyze the domains in another Actor run's dataset, e.g. the website field of a lead list or Google Maps run.

## `datasetField` (type: `string`):

Field with the domain, URL or e-mail address, e.g. "website", "domain" or "email" (dot paths work). Empty = the first URL field found (url, website, link, domain...).

## `sections` (type: `array`):

What to include. Empty = everything. The price per domain is the same whatever you choose; fewer sections make the run faster.

## `maxConcurrency` (type: `integer`):

How many websites are read at the same time.

## `siteTimeoutSecs` (type: `integer`):

A website that takes longer gets a free row (the data sections are still included).

## Actor input object example

```json
{
  "domains": [
    "stripe.com",
    "figma.com",
    "apify.com",
    "bbc.co.uk",
    "tools.yukai.uk"
  ],
  "sections": [
    "traffic",
    "authority",
    "techStack",
    "aiCrawlers",
    "domainAge",
    "hiring"
  ],
  "maxConcurrency": 8,
  "siteTimeoutSecs": 90
}
```

# Actor output Schema

## `domains` (type: `string`):

No description

## `full` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "stripe.com",
        "figma.com",
        "apify.com",
        "bbc.co.uk",
        "tools.yukai.uk"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tidytools/domain-overview").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "stripe.com",
        "figma.com",
        "apify.com",
        "bbc.co.uk",
        "tools.yukai.uk",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("tidytools/domain-overview").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "stripe.com",
    "figma.com",
    "apify.com",
    "bbc.co.uk",
    "tools.yukai.uk"
  ]
}' |
apify call tidytools/domain-overview --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tidytools/domain-overview"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/qUmyFGvHNegwY3URy/builds/GbhkcHd1Za2xCZHH5/openapi.json