# Wayback Machine Scraper: Snapshots, Archived URLs & History (`tidelane/wayback-machine`) Actor

Get every Wayback Machine snapshot of a page, every archived URL of a domain, or a history summary (first and last capture, captures per year, gaps). Date, status, file-type and URL filters, optional text of the archived pages. Uses the Internet Archive's public CDX API, no key needed.

- **URL**: https://apify.com/tidelane/wayback-machine.md
- **Developed by:** [Tidelane Data](https://apify.com/tidelane) (community)
- **Categories:** SEO tools, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.14 / 1,000 result rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Wayback Machine Scraper: Snapshots, Archived URLs & History

Get data from the **Internet Archive's Wayback Machine** in bulk, in three modes:

1. **Snapshots**: every capture of a page, with date, HTTP status, file type, size, content hash and the link to the snapshot. Optionally one per day, month or year, or only when the content changed.
2. **Archived URLs**: every URL of a domain that the archive has ever captured, each once. Recover lost pages, find old URLs for redirect maps, or map a site you are about to buy.
3. **History summary**: first and last capture, archive age in years, captures per year and years without captures. A quick check of expired domains and of a site's past.

Add the **text of archived pages** (title, description, language, readable text) to read old versions without opening a browser.

- ✅ **No API key**: uses the Internet Archive's public CDX API
- ✅ **Filters**: date range, only HTTP 200, file types (for example `text/html` or `application/pdf`), skip CSS/JS/images, URL contains
- ✅ **Big domains**: pages through results automatically (tested with 12,000 URLs of one domain in 103 seconds)
- ✅ **Reliable**: waits and retries when the archive is busy or rate-limits
- ✅ **Many targets in one run**, with a summary per target in `OUTPUT`
- ✅ **Fair billing**: per result row; history summaries and page texts are billed separately, only when they were delivered

### Input examples

Yearly snapshots of a page, with the text of the first three:

```json
{
    "targets": ["books.toscrape.com"],
    "mode": "snapshots",
    "collapse": "year",
    "includeContent": true,
    "maxContentPerTarget": 3
}
```

All archived pages and documents of a domain (subdomains included):

```json
{
    "targets": ["toscrape.com"],
    "mode": "urls",
    "excludeAssets": true,
    "maxResultsPerTarget": 10000
}
```

`targets` accepts domains, URLs, `example.com/blog/*` (everything under a path) and `*.example.com` (all subdomains).

### Output examples

Real output from test runs. A snapshot with text (`text` shortened):

```json
{
    "target": "books.toscrape.com",
    "url": "http://books.toscrape.com:80/",
    "timestamp": "2016-10-28T23:08:07Z",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "QSOKEUEELB23XUVZRYFWHGOUHOTW5WOE",
    "lengthBytes": 5269,
    "snapshotUrl": "https://web.archive.org/web/20161028230807/http://books.toscrape.com:80/",
    "rawUrl": "https://web.archive.org/web/20161028230807id_/http://books.toscrape.com:80/",
    "content": {
        "title": "All products | Books to Scrape - Sandbox",
        "description": null,
        "language": "en-us",
        "text": "Books to Scrape We love being scraped!\nHome\nAll products\nBooks\nTravel\nMystery\n…",
        "wordCount": 308,
        "truncated": false
    }
}
```

An archived URL (mode `urls`): the first capture of the URL plus a link to its latest capture:

```json
{
    "target": "toscrape.com",
    "url": "http://books.toscrape.com/catalogue/amid-the-chaos_788/index.html",
    "timestamp": "2017-09-25T11:15:22Z",
    "statusCode": 200,
    "mimeType": "text/html",
    "lengthBytes": 4131,
    "snapshotUrl": "https://web.archive.org/web/20170925111522/http://books.toscrape.com/catalogue/amid-the-chaos_788/index.html",
    "latestSnapshotUrl": "https://web.archive.org/web/http://books.toscrape.com/catalogue/amid-the-chaos_788/index.html"
}
```

A history summary (mode `summary`):

```json
{
    "target": "books.toscrape.com",
    "archived": true,
    "firstSnapshot": "2016-10-28T23:08:07Z",
    "lastSnapshot": "2026-10-11T08:46:30Z",
    "archiveAgeYears": 10,
    "totalCaptures": 303,
    "capturesPerYear": { "2016": 3, "2017": 19, "2018": 34, "2019": 23, "2020": 23, "2021": 19, "2022": 23, "2023": 23, "2024": 67, "2025": 37, "2026": 32 },
    "yearsWithoutCaptures": [],
    "firstSnapshotUrl": "https://web.archive.org/web/20161028230807/books.toscrape.com/",
    "lastSnapshotUrl": "https://web.archive.org/web/20261011084630/books.toscrape.com/",
    "allSnapshotsUrl": "https://web.archive.org/web/*/books.toscrape.com/"
}
```

`rawUrl` is the archived file exactly as it was captured, without the Wayback toolbar.

### Pricing

Per result row (snapshot or archived URL), plus a separate price per page text and per history summary. In our tests 12,000 archived URLs took 103 seconds with the default 256 MB memory.

Paid Apify plans pay less: 10% off on Starter, 20% on Scale and 30% on Business and higher plans.

### FAQ

**Why do I get fewer results than the archive shows?** By default only successful captures (HTTP 200) are returned. Turn off "Only successful captures" to also get redirects and error pages. In mode `urls` each URL appears once.

**Why are `http://` and `https://` or `:80` versions both there?** The archive stores the URL as it was captured. They are the same page; the snapshot links work for both.

**Some sites return nothing.** Site owners can ask the Internet Archive to exclude their site; then there is no public data. A target with no captures gives zero rows (or `archived: false` in the summary).

**Is the Wayback Machine sometimes down?** Yes, the archive is occasionally offline or slow. The Actor retries busy answers; if the archive stays down, the target gets an `error` in `OUTPUT` and you are not charged for it.

### More tools from Tidelane Data

- [WHOIS Domain Lookup](https://apify.com/tidelane/domain-lookup): availability, registrar and expiry dates via RDAP
- [Sitemap Extractor](https://apify.com/tidelane/sitemap-extractor): all URLs of a website from its sitemaps, with lastmod
- [Website Screenshot API](https://apify.com/tidelane/website-screenshots): full-page and mobile screenshots without cookie banners
- [Broken Link Checker](https://apify.com/tidelane/broken-link-checker): find 404s, redirects and dead links

All our tools: [apify.com/tidelane](https://apify.com/tidelane)

***

#### Impressum

Levi Griesinger, Tidelane Data\
Besigheimer Straße 18\
74369, Löchgau, Deutschland\
E-Mail: LeviGriesinger@gmail.com

# Actor input Schema

## `targets` (type: `array`):

Pages or domains to look up, for example books.toscrape.com or https://example.com/blog/. End with /\* for everything under a path.

## `mode` (type: `string`):

Snapshots: every capture of the URL(s). Archived URLs: each archived URL of a site once (first capture). History summary: first and last capture, captures per year and gaps per URL.

## `matchType` (type: `string`):

Which archived URLs belong to a target. Default: exact URL for snapshots, the whole domain with subdomains for archived URLs.

## `from` (type: `string`):

Only captures from this date on (2020, 2020-05 or 2020-05-17).

## `to` (type: `string`):

Only captures up to this date (2021, 2021-12 or 2021-12-31).

## `collapse` (type: `string`):

Snapshots mode: keep only the first capture per hour, day, month or year, or only captures where the content changed.

## `onlySuccessful` (type: `boolean`):

Skip captured redirects and error pages.

## `mimeTypes` (type: `array`):

Keep only these MIME types, for example text/html or application/pdf. Empty keeps all.

## `excludeAssets` (type: `boolean`):

Useful for archived URLs: keeps pages and documents only.

## `urlContains` (type: `string`):

Keep only archived URLs that contain this text, for example /blog/ or .pdf.

## `maxResultsPerTarget` (type: `integer`):

Stop after this many rows per URL or domain.

## `includeContent` (type: `boolean`):

For HTML captures with status 200: download the archived page and add its title, description, language and text. Billed per page.

## `maxContentPerTarget` (type: `integer`):

Limit for 'Add the text of archived pages'.

## `maxContentChars` (type: `integer`):

Longer texts are cut (the row says truncated: true).

## `timeoutSecs` (type: `integer`):

The archive can be slow for big domains. Busy answers are retried automatically.

## Actor input object example

```json
{
  "targets": [
    "books.toscrape.com"
  ],
  "mode": "snapshots",
  "collapse": "none",
  "onlySuccessful": true,
  "excludeAssets": false,
  "maxResultsPerTarget": 1000,
  "includeContent": false,
  "maxContentPerTarget": 50,
  "maxContentChars": 50000,
  "timeoutSecs": 120
}
```

# Actor output Schema

## `results` (type: `string`):

All results of the run as dataset items.

## `summary` (type: `string`):

Summary per input with status (ok, error and the reason).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "books.toscrape.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("tidelane/wayback-machine").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "targets": ["books.toscrape.com"] }

# Run the Actor and wait for it to finish
run = client.actor("tidelane/wayback-machine").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "books.toscrape.com"
  ]
}' |
apify call tidelane/wayback-machine --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tidelane/wayback-machine"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/qlDkoFJqpdAX9NOYw/builds/PbdWSn5kRMCZjHgoP/openapi.json