# Wayback Machine Scraper — $0.50/1k · Archived Pages & URLs (`tidydata/wayback-machine-scraper`) Actor

Wayback Machine (archive.org, Internet Archive) scraper: every snapshot of any website with date and status, all archived URLs of a domain, the capture closest to a date, and the text of old page versions. Bulk, no API key. $0.50 per 1,000 snapshots.

- **URL**: https://apify.com/tidydata/wayback-machine-scraper.md
- **Developed by:** [tidydata](https://apify.com/tidydata) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 snapshot or archived urls

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🏛️ Wayback Machine Scraper — Archived Pages, URLs & Snapshots

Get the **Internet Archive Wayback Machine** (web.archive.org) history of any website in bulk: **every snapshot** with date, HTTP status and file type, **all archived URLs of a domain** (including pages that no longer exist), the **capture closest to a date**, and the **title, description and text of old page versions** exactly as they were.

**$0.50 per 1,000 snapshots.** Reading the archived page's content is optional ($0.002 per page). URLs with no captures and errors are free. No API key.

- ✅ **Snapshot history** of a URL, a path, a host or a whole domain with subdomains
- ✅ **All archived URLs of a site** — one row per page, with its first capture: find **deleted pages, old blog posts and lost URLs** to redirect
- ✅ **Closest capture to any date** — "what did this page look like on 2019-05-01?"
- ✅ **Keep one capture per hour, day, month or year**, or only when the **content changed**
- ✅ **Date range**, **only successful captures (HTTP 200)** and **file type** filters (HTML pages, PDFs, images…)
- ✅ **Old page content**: title, meta description, clean text, word count and number of links — optionally the raw HTML
- ✅ Links ready to open: the Wayback page (`archiveUrl`) and the **original file without the Wayback toolbar** (`rawUrl`)
- ✅ Bulk: many URLs per run, automatic paging through large histories, polite retries when the archive is busy

### Who uses it

- **SEO specialists** — recover lost URLs and content after a migration, find old backlink targets, audit a site's history before buying an expired domain.
- **Content teams** — get back deleted articles and old versions of your own pages.
- **Competitive research** — see how competitors' pricing, homepage and messaging changed over the years.
- **Lawyers, journalists & OSINT** — dated evidence of what a page said and when.
- **AI & data teams** — historic web text for datasets and change analysis.

### How to use it

1. Add **URLs or domains** (`example.com`, `https://example.com/pricing`, or a `web.archive.org` link).
2. Choose **what to get**: snapshot history, all archived URLs of a site, or the capture closest to a date.
3. Choose the **match** (exact URL, everything under a path, the host, or the whole domain) and how many captures to keep (one per day/month/year or on content change).
4. Optional: **date range**, **file type**, and **download the archived page** to get its title and text.
5. Run and export to **CSV, Excel or JSON**, or connect it to Google Sheets, Make, Zapier or n8n.

#### Examples

Monthly history of a pricing page, with its text:

```json
{ "urls": ["https://example.com/pricing"], "mode": "snapshots", "collapse": "month", "fetchContent": "text" }
```

Every page ever archived on a domain (find lost URLs):

```json
{ "urls": ["example.com"], "mode": "urls", "matchType": "domain", "mimeType": "text/html", "maxResultsPerUrl": 50000 }
```

### Output example

```json
{
  "type": "snapshot",
  "input": "apify.com",
  "url": "https://www.apify.com/",
  "timestamp": "20171010000347",
  "date": "2017-10-10T00:03:47Z",
  "archiveUrl": "https://web.archive.org/web/20171010000347/https://www.apify.com/",
  "rawUrl": "https://web.archive.org/web/20171010000347id_/https://www.apify.com/",
  "statusCode": 200,
  "mimeType": "text/html",
  "digest": "3JVHMXDJHHC33YHJT32EGULLRQHVSGI3",
  "lengthBytes": 9871,
  "title": "Apify - The web automation platform",
  "metaDescription": null,
  "text": "Apify - The web automation platform\nLibrary\nHelp\nPricing…",
  "wordCount": 1073
}
```

`digest` changes only when the archived content changed — handy to detect edits. Rows of type `not_archived` (free) tell you when nothing matched.

### Pricing

- **$0.0005 per snapshot or archived URL** ($0.50 per 1,000)
- **$0.002 per archived page downloaded** (only when "Download the archived page" is on)

Not-archived URLs and errors are free. Higher Apify plans get an automatic discount, and you can set a **maximum cost per run**.

### Tips

- Combine with [Sitemap URL Extractor](https://apify.com/tidydata/sitemap-url-extractor) to compare today's pages with the archived ones and spot what disappeared.
- Check which old URLs still work with [URL Status & Redirect Checker](https://apify.com/tidydata/url-status-checker).

### FAQ

**Why are some captures missing?**
The Wayback Machine only has what its crawlers (and users) saved. Use `matchType: prefix` or `domain` to include URL variants, and remove the date range if you get nothing.

**What does "only successful captures" do?**
It skips captures stored as redirects, 404s or server errors. Turn it off to see them too.

**Is the text exactly what the page showed?**
It's the HTML the archive stored on that date, read without the Wayback toolbar. Pages that built their content with JavaScript may show little text.

**How fast is it?**
Hundreds of snapshots per minute for histories; downloading page content depends on the archive's speed (a few pages per second). The Actor slows down automatically when the archive asks it to.

**Is it legal?**
It uses the Internet Archive's public CDX and Availability APIs. The archived pages belong to their owners — respect copyright when you reuse them.

Something doesn't work? Open an issue in the **Issues** tab with the URL — we answer fast.

# Actor input Schema

## `urls` (type: `array`):

One per line: example.com, https://example.com/pricing or a web.archive.org link.

## `mode` (type: `string`):

'All archived URLs' finds old and deleted pages of a site — set 'Match' to 'Whole domain' or 'Everything under this path'.

## `matchType` (type: `string`):

How the URL is matched in the archive.

## `collapse` (type: `string`):

Reduces repeated captures. 'Content change' keeps only captures whose content differs from the previous one.

## `fromDate` (type: `string`):

YYYY, YYYY-MM or YYYY-MM-DD.

## `toDate` (type: `string`):

YYYY, YYYY-MM or YYYY-MM-DD.

## `targetDate` (type: `string`):

For 'The capture closest to a date'. YYYY-MM-DD. Empty = today (latest capture).

## `onlySuccessful` (type: `boolean`):

Skips redirects, 404s and errors stored in the archive.

## `mimeType` (type: `string`):

e.g. text/html for web pages, application/pdf for PDFs, image/jpeg for photos.

## `maxResultsPerUrl` (type: `integer`):

Oldest first.

## `fetchContent` (type: `string`):

Reads each archived version as it was on that date. Charged separately per page.

## Actor input object example

```json
{
  "urls": [
    "apify.com"
  ],
  "mode": "snapshots",
  "matchType": "exact",
  "collapse": "month",
  "onlySuccessful": true,
  "maxResultsPerUrl": 100,
  "fetchContent": "none"
}
```

# Actor output Schema

## `results` (type: `string`):

One row per archived capture or URL.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "apify.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tidydata/wayback-machine-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": ["apify.com"] }

# Run the Actor and wait for it to finish
run = client.actor("tidydata/wayback-machine-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "apify.com"
  ]
}' |
apify call tidydata/wayback-machine-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tidydata/wayback-machine-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/sgLSRMVJ6UwAqgRW8/builds/IcfklNe04yiMJazQs/openapi.json