# Bulk Domain History Checker (Wayback Machine) (`lindenbyte/wayback-domain-history`) Actor

Check the Wayback Machine history of many domains at once: first/last capture, archive age, captures per year, gaps, status codes, redirects to other sites, and archived homepage titles per year with spam, parked and language-change flags. For expired-domain due diligence and SEO.

- **URL**: https://apify.com/lindenbyte/wayback-domain-history.md
- **Developed by:** [Lindenbyte Studio](https://apify.com/lindenbyte) (community)
- **Categories:** SEO tools, Developer tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk Domain History Checker (Wayback Machine)

Check the **Wayback Machine history of hundreds of domains at once** and get **one clean summary row per domain**: first and last capture, archive age, years active, captures per year, the longest dark period, HTTP status mix, **where the domain redirected to over the years**, and the **archived homepage title for each year** with automatic flags for **past spam (casino, pharma, adult, replica), parked / for-sale pages and language changes**.

Built for **expired-domain due diligence**, SEO audits and research. No login, no API key, no browser, no proxy.

### Why this one

- **One row per domain, not thousands of raw snapshots.** Most Wayback scrapers dump every capture and leave the analysis to you. This Actor does the analysis and still gives you direct Wayback links for every era.
- **Homepage titles per year (the killer check).** It opens archived copies of the homepage (one per year, spread over the whole history) and reads `<title>`, meta description and `<html lang>`. A domain that was a Japanese replica-watch shop in 2016 or a casino affiliate in 2019 shows up immediately.
- **Spam, parked and language flags.** `riskFlags` gives a one-glance verdict: `spam-keywords-in-titles`, `parked-or-for-sale`, `language-change`, `redirected-elsewhere`, `long-gap`, `never-archived`, `never-served-200`.
- **Redirect history.** For years when the homepage was a 301/302 (and for meta-refresh pages) it records the target, and lists `externalRedirectDomains` (redirects to a different site are a classic sign of a past sale, merger or a PBN).
- **Correct encodings.** Old Cyrillic, Japanese, Chinese and Korean sites (windows-1251, Shift_JIS, GBK, EUC-KR...) come out readable.
- **Never-archived domains are an answer, not an error.** You get a row with `archived: false` and zero captures.
- **Failed lookups are free.** If the Wayback Machine cannot answer for a domain, you still get a row with an `error` field, and it is not charged.
- **Optional archived URL list** per domain (distinct pages ever captured, with first-capture date and Wayback link), optionally only real HTML pages that returned 200.

### Use cases

- **Expired / auction domain buying**: check age, continuity and past use before you bid (GoDaddy Auctions, DropCatch, NameJet, SnapNames lists).
- **SEO and link building**: spot domains with a spam past or a long dark period before using them, vet PBN or guest-post sites.
- **Brand and M\&A research**: when did a site launch, rebrand, redirect, go dark?
- **Academic and journalism research**: archive coverage per year for a list of sites.

### Input

| Field | What it does | Default |
|---|---|---|
| `domains` | Domains or any URL on them (`example.com`, `https://www.example.com/page`). `www.` is ignored, duplicates removed | required |
| `fetchTitles` | Read archived homepage titles (spam / language / parked checks) | `true` |
| `maxTitleFetches` | Max archived homepages read per domain (years spread evenly, first and last always included) | `12` |
| `maxRedirectChecks` | Max redirect captures resolved per domain (0 = off) | `6` |
| `countAllCaptures` | Also count every single homepage capture (`totalCaptures`). Slower for huge sites | `false` |
| `includeUrls` | Add one row per distinct archived URL of the domain (incl. subdomains) | `false` |
| `maxUrlsPerDomain` | Cost cap for the URL list | `1000` |
| `onlyHtmlUrls` | URL list: only HTML pages that returned 200 | `false` |
| `fromYear`, `toYear` | Only consider captures in this year range | all |

```json
{
  "domains": ["example.com", "geocities.com", "some-expired-domain.net"],
  "fetchTitles": true,
  "maxTitleFetches": 12
}
```

### Output

One `domain` row per domain (real example, shortened):

```json
{
  "rowType": "domain",
  "domain": "geocities.com",
  "archived": true,
  "firstCapture": "1996-10-22T17:32:45Z",
  "lastCapture": "2026-11-08T08:42:08Z",
  "archiveAgeYears": 30,
  "activeYears": 31,
  "missingYears": [],
  "longestGapMonths": 6.7,
  "captureDays": 5401,
  "capturesPerYear": { "1996": 7, "1997": 9, "2000": 131, "2010": 153, "2025": 357 },
  "capturesPerYearUnit": "days with at least one homepage capture",
  "statusCodes": { "2xx": 1015, "3xx": 3317, "4xx": 179, "5xx": 4, "other": 886 },
  "lastStatusCode": 301,
  "firstHttpsCapture": "2021-01-28T01:49:48Z",
  "titleSamples": [
    { "year": 1996, "title": "Welcome to GeoCities Home Page", "htmlLang": null, "script": "latin", "link": "https://web.archive.org/web/19961022173245/http://www.geocities.com:80/" },
    { "year": 2007, "title": "Yahoo! GeoCities: Get a free web site with easy-to-use site building tools", "script": "latin" }
  ],
  "redirects": [
    { "year": 1999, "statusCode": 301, "target": "http://geocities.yahoo.com/home/", "external": true },
    { "year": 2015, "statusCode": 200, "via": "meta-refresh", "target": "https://smallbusiness.yahoo.com/webhosting?source=geocities", "external": true },
    { "year": 2026, "statusCode": 301, "target": "https://www.yahoo.com/", "external": true }
  ],
  "externalRedirectDomains": ["geocities.yahoo.com", "smallbusiness.yahoo.com", "yahoo.com"],
  "possibleSpamKeywords": [],
  "parkedOrForSaleYears": [],
  "languagesSeen": [],
  "scriptsSeen": ["latin"],
  "languageChanges": [],
  "riskFlags": ["redirected-elsewhere"],
  "waybackLink": "https://web.archive.org/web/*/geocities.com",
  "error": null
}
```

Key fields:

- `captureDays` / `capturesPerYear`: days with at least one homepage capture (fast, bounded, comparable between domains). Turn on `countAllCaptures` for the exact raw count in `totalCaptures`.
- `statusCodes`, `mimetypes`: per capture day. `other` is mostly `warc/revisit` records (the crawler saw identical content again).
- `contentVersions`: distinct homepage contents seen (by content hash).
- `longestGapMonths` + `longestGapFrom`/`To`: the longest period with no homepage capture, a common sign that a domain dropped.
- `spamSignals`: `{ keyword, category, years }` per matched keyword (`gambling`, `pharma`, `adult`, `counterfeit`, `finance-spam`, `seo-spam`).
- `languageChanges`: `{ year, from, to }` when the `<html lang>` or the writing system of the title changed.

With `includeUrls`, extra `url` rows: `{ rowType: "url", domain, url, firstCapture, statusCode, mimetype, waybackLink }`. Use the **Archived URLs** dataset view or filter by `rowType`.

### Pricing

Pay per result: a small fee per domain summary row, a fraction of a cent per archived URL row, plus a tiny start fee per run. Domains that fail are not charged. Set a maximum charge per run in the Console and the Actor stops cleanly when it is reached.

### Speed and limits (honest)

- The Wayback Machine's CDX API is free and rate limited, so this Actor deliberately goes slowly and politely: about **30-60 seconds per domain** with titles on, **5-15 seconds** with titles and redirects off. 100 domains with defaults take roughly an hour; schedule it and connect an integration.
- Capture statistics are for the **homepage** (`domain.com/`, any www/http/https variant). The optional URL list covers the whole domain and subdomains.
- For extremely large domains (millions of URLs) the Wayback URL listing can time out; the summary row is still produced with `urlsError` explaining it.
- Some archived redirects cannot be resolved (Wayback replays them as the target page); those are reported as same-site with a `note`.
- Spam detection is a **keyword signal on titles and descriptions**, not a verdict. Always open the `link` of a flagged year to see for yourself.
- Results reflect what the Internet Archive captured. Sites excluded from the archive appear as never archived.

### FAQ

**Do I need an Internet Archive account or API key?** No.
**Can it check domain registration (WHOIS) age?** No, this is archive history. Archive age is usually the better signal for SEO because it shows when the domain actually had a site.
**Does it download whole sites?** No. It reads the archive index plus at most one archived homepage per year.

Data source: the public Internet Archive Wayback Machine CDX API. Please support the Internet Archive (archive.org/donate). Not affiliated with the Internet Archive.

# Actor input Schema

## `domains` (type: `array`):

Domains to check (example.com) or any URL on them (https://www.example.com/page): the domain is extracted, 'www.' is ignored and duplicates are removed. One summary row per domain.

## `fetchTitles` (type: `boolean`):

Opens archived copies of the homepage (at most one per year, up to the limit below) and records the <title>, meta description and <html lang>. Used to flag past spam (casino, pharma, adult, replica...), language changes, parked/for-sale pages and meta-refresh redirects. Turn off for the fastest, date-only check.

## `maxTitleFetches` (type: `integer`):

Years are spread evenly over the archive history (first and last year always included). More = finer history but slower (about 1-2 seconds each).

## `maxRedirectChecks` (type: `integer`):

For years where the archived homepage was a 3xx redirect, reads where it pointed to (one capture per year, spread evenly). Reveals domains that were redirected to another site (a common sign of a past sale, merger or a PBN). 0 = off.

## `countAllCaptures` (type: `boolean`):

By default captures are counted as 'days with at least one homepage capture' (fast and bounded). Turn on to also count every single homepage capture (totalCaptures). For very popular sites (millions of captures) the count stops at about 2 million and is marked as a lower bound.

## `includeUrls` (type: `boolean`):

Adds one row per distinct archived URL on the domain and its subdomains (first capture date, status, type, Wayback link), useful to see which pages existed or to rebuild content. Charged per URL row. The Wayback CDX API can time out listing very large domains; the summary row is still produced.

## `maxUrlsPerDomain` (type: `integer`):

Only with 'Also list archived URLs'. Cost cap: at most this many URL rows per domain (in Wayback index order: alphabetical by URL, subdomains after the main host).

## `onlyHtmlUrls` (type: `boolean`):

Only with 'Also list archived URLs'. Skips images, scripts, CSS, redirects and errors so the list contains real pages.

## `fromYear` (type: `integer`):

Optional: only consider captures from this year onwards (for example 2015).

## `toYear` (type: `integer`):

Optional: only consider captures up to and including this year.

## Actor input object example

```json
{
  "domains": [
    "example.com",
    "geocities.com"
  ],
  "fetchTitles": true,
  "maxTitleFetches": 12,
  "maxRedirectChecks": 6,
  "countAllCaptures": false,
  "includeUrls": false,
  "maxUrlsPerDomain": 1000,
  "onlyHtmlUrls": false
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "domains": [
        "example.com",
        "geocities.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("lindenbyte/wayback-domain-history").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "domains": [
        "example.com",
        "geocities.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("lindenbyte/wayback-domain-history").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "domains": [
    "example.com",
    "geocities.com"
  ]
}' |
apify call lindenbyte/wayback-domain-history --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,lindenbyte/wayback-domain-history"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/1GFZMqMbSHNghZsyC/builds/qNJncYdPIzgtyhqUc/openapi.json
