# Wayback Machine Scraper: Domain Age, Archived URLs & Content (`classier_wholemeal/wayback-machine-toolkit`) Actor

Wayback Machine scraper and Internet Archive API for bulk domain age checks, snapshot timelines, every archived URL with live 404 detection, and archived page content as clean text/Markdown. No API key, no login, no proxy.

- **URL**: https://apify.com/classier_wholemeal/wayback-machine-toolkit.md
- **Developed by:** [JackKnifeAI](https://apify.com/classier_wholemeal) (community)
- **Categories:** SEO tools, Developer tools, Open source
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.01 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Wayback Machine Scraper: Domain Age Checker, Archived URL Extractor, Dead Page Finder & Internet Archive API

A **Wayback Machine scraper** and **Internet Archive API** alternative. Get Wayback Machine data in bulk — domain age, snapshots, archived URLs and page content — without an API key:

- 🕰️ **Domain age checker**: first and last archive date, site age in years, and the years the site was active, for hundreds of domains at once.
- 📈 **Snapshot history and change timeline**: every capture of a URL, or **only the captures where content changed**, with HTTP status, content hash and a direct archive link.
- 🔗 **All archived URLs of a website + dead page finder**: every distinct URL the Wayback Machine has seen for a domain. You can also **check each one on the live site** to find pages that now return 404, error out or redirect. Use it for SEO redirect mapping, backlink recovery and site migrations.
- 📝 **Archived page content as clean text and Markdown**: the version of a page closest to any date, with title, meta description, H1s, word count, plain text and Markdown. It's ready for LLMs, RAG, research, legal evidence and competitor analysis.

### Who uses it

| Use case | Mode |
|---|---|
| Domain investors, M\&A and due diligence: how old is this site really? | `domainAge` |
| Lead scoring: established businesses vs. brand-new sites | `domainAge` |
| SEO: find old URLs that now 404 and lost backlinks, and build a 301 redirect map | `archivedUrls` + `checkLiveStatus` |
| Competitor monitoring: when did their pricing or homepage change? | `snapshots` with `onlyChanges` |
| Content recovery after a lost CMS or expired domain | `archivedUrls` → `pageContent` |
| AI / RAG datasets of historical web pages | `pageContent` |
| OSINT, journalism, legal: what did a page say on a given date? | `pageContent` with `snapshotDate` |

Search terms this listing covers: Wayback Machine scraper, Internet Archive API, domain age checker, archived page downloader, website history checker, dead page finder, CDX API.

### How to use

1. Pick a **mode**.
2. Add **domains or URLs** (`example.com` or `https://example.com/pricing`).
3. Optionally set dates, a status filter, or live checks.
4. Run, then download results as JSON, CSV, Excel or HTML, or get them through the Apify API.

#### Example input

```json
{
  "mode": "archivedUrls",
  "targets": ["example.com"],
  "htmlOnly": true,
  "checkLiveStatus": true,
  "maxItemsPerTarget": 5000
}
```

#### Example output

**Domain age**

```json
{
  "type": "domainAge",
  "domain": "apify.com",
  "archived": true,
  "firstCapture": "2007-05-31T10:15:38+00:00",
  "lastCapture": "2026-10-31T18:01:16+00:00",
  "ageYears": 19.43,
  "activeYears": ["2007", "2008", "2011", "…", "2026"],
  "activeYearCount": 18,
  "firstArchiveUrl": "https://web.archive.org/web/20070531101538/http://www.apify.com:80/"
}
```

**Archived URL + live check**

```json
{
  "type": "archivedUrl",
  "url": "https://jackknife.io/.well-known/ai-plugin.json",
  "firstCapture": "2025-10-17T02:46:17+00:00",
  "statusCode": "404",
  "liveStatus": 404,
  "isDead": true,
  "archiveUrl": "https://web.archive.org/web/20251017024617/https://jackknife.io/.well-known/ai-plugin.json"
}
```

**Page content**

```json
{
  "type": "pageContent",
  "url": "https://news.ycombinator.com/",
  "timestamp": "2015-01-02T02:00:25+00:00",
  "title": "Hacker News",
  "wordCount": 589,
  "text": "Hacker News\nnew | comments | …",
  "markdown": "…"
}
```

### Input reference

| Field | Description |
|---|---|
| `mode` | `domainAge`, `snapshots`, `archivedUrls` or `pageContent` |
| `targets` | Domains or URLs |
| `fromDate` / `toDate` | `YYYY`, `YYYYMM` or `YYYYMMDD` (snapshots, archivedUrls) |
| `onlyChanges` | Snapshots: skip captures with identical content (default `true`) |
| `statusFilter` | Keep only captures with this archived HTTP status, e.g. `200` |
| `htmlOnly` | archivedUrls: HTML pages only (default `true`) |
| `checkLiveStatus` | archivedUrls: check each URL on the live site now |
| `snapshotDate` | pageContent: capture closest to this date (default: latest) |
| `maxItemsPerTarget` | Limit per domain or URL (default 1000) |
| `maxConcurrency` | Parallel requests (default 3; please be gentle with archive.org) |

### Pricing (pay per event)

You pay only for results. There's no subscription and no proxy cost.

| Event | Price |
|---|---|
| Domain age summary | $4 per 1,000 domains |
| Snapshot record | $0.50 per 1,000 snapshots |
| Archived URL (incl. live check) | $0.80 per 1,000 URLs |
| Archived page content (text + Markdown) | $4 per 1,000 pages |

Set a **maximum charge per run** and the Actor stops cleanly when it hits it. Failed targets are reported with `type: "error"` and are never charged.

### FAQ

**Is this legal?** Yes. The Actor only uses the Internet Archive's public, documented endpoints (the CDX server and raw snapshots). It doesn't log in or bypass anything, and it identifies itself with a clear User-Agent. Archived content belongs to its original owners, so make sure your use is lawful (research, SEO, due diligence and similar).

**Why is a domain "not archived"?** The Wayback Machine has no captures of its homepage. That usually means a brand-new or never-public site.

**Why do some snapshots show `contentChanged: true` very often?** Some sites put timestamps or tokens in every page, and `http`, `https` and `www` variants can differ. Use `statusFilter: "200"` and target the exact URL you care about.

**Is there an API or integration?** Yes. You can run it through the Apify API, Make, Zapier, n8n or LangChain, or call it as a tool from AI agents through the Apify MCP server.

***

*Not affiliated with the Internet Archive. If you rely on the Wayback Machine, please consider [donating to the Internet Archive](https://archive.org/donate).*

# Actor input Schema

## `mode` (type: `string`):

domainAge = first/last archive date, site age and active years per domain. snapshots = capture history of a URL (optionally only when content changed). archivedUrls = every distinct URL ever archived for a domain, optionally checked live to find dead (404) pages. pageContent = archived page as clean text and Markdown at a chosen date.

## `targets` (type: `array`):

Domains (example.com) or full URLs. One per line.

## `fromDate` (type: `string`):

Only captures on/after this date. Format YYYY, YYYYMM or YYYYMMDD. Used by snapshots and archivedUrls.

## `toDate` (type: `string`):

Only captures on/before this date. Format YYYY, YYYYMM or YYYYMMDD.

## `onlyChanges` (type: `boolean`):

snapshots mode: skip consecutive captures with identical content (uses the archive's content hash). Turns thousands of captures into a clean change timeline.

## `statusFilter` (type: `string`):

Keep only captures whose archived HTTP status matches, e.g. 200 or 404. Leave empty for all.

## `htmlOnly` (type: `boolean`):

archivedUrls mode: skip images, CSS, JS and other non-HTML files.

## `checkLiveStatus` (type: `boolean`):

archivedUrls mode: request each archived URL on the live site and report its current HTTP status and redirect target, so you can spot pages that now 404 and need redirects or content recovery.

## `snapshotDate` (type: `string`):

pageContent mode: get the capture closest to this date (YYYYMMDD). Leave empty for the latest capture.

## `maxItemsPerTarget` (type: `integer`):

Upper limit of snapshots or URLs returned for each target.

## `maxConcurrency` (type: `integer`):

Parallel requests. Keep low; the Internet Archive rate-limits aggressive clients.

## Actor input object example

```json
{
  "mode": "domainAge",
  "targets": [
    "apify.com",
    "example.com"
  ],
  "onlyChanges": true,
  "htmlOnly": true,
  "checkLiveStatus": false,
  "maxItemsPerTarget": 1000,
  "maxConcurrency": 3
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "apify.com",
        "example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("classier_wholemeal/wayback-machine-toolkit").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "targets": [
        "apify.com",
        "example.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("classier_wholemeal/wayback-machine-toolkit").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "apify.com",
    "example.com"
  ]
}' |
apify call classier_wholemeal/wayback-machine-toolkit --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,classier_wholemeal/wayback-machine-toolkit"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/TVHFFscW5YaCjh1yB/builds/ywzjfnge7gJzUxEOK/openapi.json