# Wayback Machine Scraper (`spookyweb/wayback-machine-scraper`) Actor

Every archived capture of any URL, from the Internet Archive. List snapshots for one page, a path prefix or a whole domain, filter by status code, MIME type and date, and pull back the archived HTML exactly as it was originally served.

- **URL**: https://apify.com/spookyweb/wayback-machine-scraper.md
- **Developed by:** [丂卩ㄖㄖҜㄚ](https://apify.com/spookyweb) (community)
- **Categories:** SEO tools, News, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.04 / 1,000 snapshot fetched with contents

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Wayback Machine Scraper

Every archived capture of any URL from the Internet Archive, what changed between them, and the page itself exactly as it was originally served.

### 🔍 What does Wayback Machine Scraper do?

It queries the Internet Archive's index for a URL and returns one row per capture, with the option to fetch the archived page itself and compare it with the capture before it.

- **One page, a path, a hostname or a whole domain**, and up to 50 URLs a run.
- **The original bytes.** Content is fetched with the archive's `id_` modifier,
  so you get the HTML the server actually sent, with no archive toolbar and no
  rewritten links. Extracted text is returned alongside it, not instead of it.
- **Change detection.** Each capture is compared with the one before it using
  the archive's own content fingerprint, so finding out *whether* a page moved
  costs nothing extra. Turn on content retrieval and you also get *what* moved.
- **Only the changes.** A decade of daily captures collapses to the handful of
  dates the page actually changed.
- **Point in time evidence.** Ask for the capture closest to a date and get it
  with the gap reported in days.
- **Monitoring.** Remembers the newest capture per URL and returns only what is
  new since the last run, so a schedule reports activity rather than history.
- **Presets** for SEO history, compliance evidence, competitor watch and
  forensics, so you do not have to work the options out.

Server side filtering by status code, MIME type and date range, so the archive
does the narrowing rather than you paying to download rows you then discard.

### 📊 What data can I extract from the Wayback Machine?

One record per capture:

| Field | What it is |
|---|---|
| `timestamp`, `capturedAt` | Capture time, raw and as ISO 8601 UTC |
| `url` | The URL as captured |
| `statusCode`, `mimeType` | What it was served as, or `null` if unrecorded |
| `digest`, `lengthBytes` | Archive content fingerprint and size |
| `snapshotUrl`, `rawUrl` | Viewable archive link, and the original bytes link |
| `html` | The page as originally served, when content is retrieved |
| `text` | Readable text, scripts, styles and archive toolbar removed |
| `changed`, `changeType` | Whether it differs from the previous capture, and how |
| `changeBasis` | `digest`, `length` or `unknown`, so you can judge the claim |
| `bytesDelta` | Size difference from the previous capture |
| `diff` | Score, magnitude, categories, title and price changes, sample text |
| `distanceFromTargetDays` | Days from the requested date, on closest-date runs |

### 💡 Why use the Wayback Machine Scraper?

**Point in time evidence.** Prove what a page said on a particular date, with the capture time and the archive link attached.

**Competitor watch.** Track a competitor's pricing, claims or terms over time and see only the dates they moved.

**Content recovery.** Recover a page that was taken down or rewritten, and feed the archived HTML into your own parsing.

**Migration repair.** Rebuild a URL inventory after a migration lost the old sitemap.

### 🚀 How do I use Wayback Machine Scraper?

1. Click **Try for free**.
2. Put the page, path, hostname or domain you want into `url`, or a list into `urls`.
3. Set `matchType` to decide how wide that goes, and narrow with `from`, `to`, `statusCodes` and `mimeTypes`.
4. Turn on `fetchContent` if you want the archived pages themselves, then click **Start**.
5. Download the results as JSON, CSV or Excel, or pull them from the API.

### ⬇️ Input

```json
{
  "url": "bbc.co.uk/news",
  "from": "2024",
  "statusCodes": ["200"],
  "mimeTypes": ["text/html"]
}
```

| Field | Type | Default | What it does |
|---|---|---|---|
| `url` | string | `bbc.co.uk/news` | One page, path, hostname or whole domain |
| `urls` | array | | Up to 50 URLs in one run |
| `useCase` | string | | Preset for SEO history, compliance evidence, competitor watch or forensics |
| `matchType` | string | `exact` | How to match the URL, exact, prefix, host or domain |
| `from` | string | `2024` | Earliest capture date or year |
| `to` | string | | Latest capture date or year |
| `targetDate` | string | | Return the capture closest to this date, with the gap in days |
| `maxSnapshots` | integer | `100` | Cap on captures returned |
| `statusCodes` | array | `200` | Only captures with these status codes |
| `mimeTypes` | array | `text/html` | Only captures with these MIME types |
| `detectChanges` | boolean | `false` | Compare each capture with the one before it |
| `onlyChanged` | boolean | `false` | Return only the captures where something moved |
| `fetchContent` | boolean | `false` | Retrieve the original page bytes and text |
| `maxContentFetches` | integer | `25` | Cap on how many captures are retrieved in full |
| `monitor` | boolean | `false` | Return only captures newer than the last run |

### ⬆️ Output

#### Table view

Results arrive as a Snapshots table you can sort and filter in the Console, with the capture time, URL, status code, MIME type, retrieved content status, text length and the archive link lined up per capture.

#### JSON

A typical row:

```json
{
  "capturedAt": "2024-01-04T21:34:53.000Z",
  "url": "https://stripe.com/pricing",
  "statusCode": 200,
  "mimeType": "text/html",
  "contentStatus": 200,
  "textChars": 23837,
  "snapshotUrl": "https://web.archive.org/web/20240104213453/https://stripe.com/pricing",
  "timestamp": "20240104213453"
}
```

Download it from the run as JSON, CSV or Excel, or read it straight from the API.

### How the change detection decides

`changed` comes from the archive's own content digest, which is computed over
the stored bytes and is the authoritative signal. Where a row has no digest,
size is used instead, and `changeBasis` says so, because that is a weaker claim
and you should be able to see which one you got. Where neither exists, nothing
is claimed.

`diff` needs both pages, so it only appears where content was retrieved. It is
scored on the share of words that moved rather than raw counts, so a paragraph
edit on a short page is not ranked below a footer tweak on a long one.

**A title or price change is never reported as minor.** Both can move while
almost no words do, and both are usually the reason someone is watching.

A capture can be `changed: true` with a `diff` of `none`. That is not a
contradiction: the stored bytes differ but the readable text does not, so the
change was in markup or scripts rather than in anything a reader sees.

### Notes on accuracy

**A URL with no captures is an answer, not an error.** The run finishes with
zero results and says what to widen. It does not fail.

**Filters are checked, not assumed.** Every returned row is tested against the
filters that were requested, and the run warns if any row does not satisfy them
rather than presenting the wrong answer with confidence.

**Timestamps come from the index, never invented.** A made up timestamp answers
with an empty redirect, which is indistinguishable from a page that archived to
nothing, so content is only fetched at a timestamp the archive returned.

**The archive does not replay original response headers.** If you are detecting
technologies from archived pages, anything visible only in a response header,
such as the CDN, web server or language, is not recoverable from an archived
capture by anyone. Content in the HTML is unaffected.

**Coverage is the archive's, not ours.** Archive.org crawls on its own
schedule, so not every change is captured.

### 💰 How much does it cost?

Pay per event, so a run costs what it does.

| Event | Price |
|---|---|
| Snapshot indexed | $0.003 |
| Snapshot fetched with content | $0.0034 |

Fetching replaces the indexing charge for that capture rather than adding to
it. Indexing 100 captures costs $0.30. Indexing 100 and retrieving 25 of them
costs $0.31.

Use `maxSnapshots` to cap the index side and `maxContentFetches` to cap the retrieval side.

### 🔌 Integrations

Send results straight to Google Sheets, Slack, Airtable, Zapier, Make or your own webhook using [Apify integrations](https://docs.apify.com/platform/integrations). You can also trigger a run whenever something happens in another tool.

### 🔗 Using Wayback Machine Scraper with the Apify API

```bash
curl -X POST "https://api.apify.com/v2/acts/spookyweb~wayback-machine-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "bbc.co.uk/news", "from": "2024", "statusCodes": ["200"], "mimeTypes": ["text/html"]}'
```

Or with the Apify client:

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('spookyweb/wayback-machine-scraper').call({
  url: 'bbc.co.uk/news',
  from: '2024',
  statusCodes: ['200'],
  mimeTypes: ['text/html'],
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
```

Full detail is in the [Apify API reference](https://docs.apify.com/api/v2), and every run is also callable from the [Python](https://docs.apify.com/api/client/python) and [JavaScript](https://docs.apify.com/api/client/js) clients.

### ❓ FAQ

#### What is a capture, and how do I pick one?

A capture is one archived copy of a URL at one moment, identified by its timestamp. A busy page can have thousands. Narrow with `from` and `to`, filter to the status codes and MIME types you care about, or set `targetDate` to get the capture closest to a particular day, with `distanceFromTargetDays` telling you how far off it landed.

#### What does change detection do?

It compares each capture with the one before it and sets `changed`, `changeType` and `changeBasis`. The comparison uses the archive's own content digest, so it costs nothing extra, and `onlyChanged` collapses a decade of daily captures down to the handful of dates the page actually moved.

#### Can I get the page content as well as the index?

Yes. Turn on `fetchContent` and each capture carries `html`, the bytes the server originally sent with no archive toolbar and no rewritten links, plus `text`, the readable content with scripts and styles removed. `maxContentFetches` caps how many are retrieved so a wide index run does not turn into a wide download.

#### What does matchType do?

It decides how wide the URL query goes. `exact` returns captures of that one URL. `prefix` returns everything under that path. `host` returns everything on that exact hostname. `domain` returns the hostname and its subdomains. Widening it is the fastest way to turn one page into a whole site inventory.

#### Can I monitor a page for changes over time?

Yes. Set `monitor` and run it on a schedule. The Actor remembers the newest capture per URL and returns only what is new since the last run, so each run reports activity rather than repeating the history. Combine it with `detectChanges` and `onlyChanged` to be told only when something actually moved.

#### Why did my run return nothing?

Because the archive has no captures matching what you asked for. That is an answer rather than an error, so the run finishes successfully with zero results and says what to widen. Try a broader `matchType`, an earlier `from`, or dropping the status code and MIME filters.

### ⚖️ Is it legal to scrape the Wayback Machine?

This reads the Internet Archive's public Wayback Machine through its published CDX API. The archive exists to make historical web pages available to the public, and this Actor respects its rate limits.

Scraping publicly available data is legal in the UK, the EU and the US. What you do with an archived page is still governed by the copyright and terms of the original site, so treat archived content as you would the live version. If a capture contains personal data, handling it is on you under GDPR. Apify's [ethical scraping guide](https://blog.apify.com/is-web-scraping-legal/) covers the wider picture.

### 👍 Your feedback

Found a bug, or want a field that is not here yet? Open an issue on the Actor's Issues tab. Requests that make the data more useful get built, and problems get fixed quickly.

### 🔎 You might also like

| Actor | What it does |
|---|---|
| [Website Contact Scraper](https://apify.com/spookyweb/website-contact-scraper) | Emails, phones, socials and addresses from company websites, one record per domain |
| [Company Email Finder](https://apify.com/spookyweb/company-email-finder) | Published company addresses, the naming pattern behind them and an MX check |
| [Telegram Channel Scraper](https://apify.com/spookyweb/telegram-channel-scraper) | Posts, views and reactions from public Telegram channels, no account needed |

# Actor input Schema

## `url` (type: `string`):

The page or site to look up in the Internet Archive. The protocol is optional and is ignored, so bbc.co.uk and https://bbc.co.uk are the same query.

## `urls` (type: `array`):

Look up several pages or sites in one run, up to 50. Takes priority over the single URL field when both are given, and each URL is charged separately.

## `useCase` (type: `string`):

Applies the options that suit a job so you do not have to work them out. seo crawls a path prefix monthly, compliance keeps only captures whose content changed and fetches them, competitor watches a page for changes and returns only those, forensics returns every capture across a whole domain. Anything you set explicitly wins over the preset.

## `matchType` (type: `string`):

exact returns captures of that one URL. prefix returns everything under that path. host returns every page on that exact hostname. domain also includes subdomains such as news.bbc.co.uk.

## `from` (type: `string`):

Only return captures on or after this date. Accepts a year, a year and month, or a full date: 2024, 202403 or 20240315.

## `to` (type: `string`):

Only return captures on or before this date. Same formats as the earliest capture: 2024, 202403 or 20240315.

## `targetDate` (type: `string`):

Return only the single capture nearest this date, with the gap reported in days. This is the one to use for evidence of what a page said on a particular day, such as the date a contract was signed.

## `snapshotsPer` (type: `string`):

A busy homepage holds tens of thousands of near identical captures. Keeping one per day, month or year gives a usable history instead. Content changes only keeps a capture when the archive's own content fingerprint differs from the one before it, which is the cheapest way to find the moments a page actually moved.

## `maxSnapshots` (type: `integer`):

Stop after this many captures for each URL. Every snapshot returned is charged, so this is the ceiling on what a run can cost.

## `statusCodes` (type: `array`):

Only return captures served with these status codes. Use 200 to exclude redirects and error pages, which is usually what you want.

## `mimeTypes` (type: `array`):

Only return captures of these content types, for example text/html to exclude images, scripts and stylesheets.

## `detectChanges` (type: `boolean`):

Compare each capture with the one before it and label what moved. Uses the archive's own content fingerprint, so whether a page changed costs nothing extra. What changed needs the content, so turn on retrieving it as well for a full diff.

## `onlyChanged` (type: `boolean`):

Drop captures identical to the one before them, so a decade of history collapses to the handful of dates the page actually moved. Needs change detection switched on.

## `fetchContent` (type: `boolean`):

Download each capture as well as indexing it. Charged at the higher rate, and it is what enables the full diff including title and price changes.

## `includeRawHtml` (type: `boolean`):

Return the page exactly as the server originally sent it, with no archive toolbar and no rewritten links. This is the original bytes rather than a cleaned copy, so it can be parsed as if you had fetched the page at the time.

## `includeText` (type: `boolean`):

Return the readable text of the page alongside the HTML, with scripts, styles and the archive's own toolbar removed. Useful for reading and for feeding to a model.

## `maxTextChars` (type: `integer`):

Cut the extracted text at this many characters. Leave at 0 to keep the whole page, which is the default because truncation loses the end of long documents silently.

## `maxContentFetches` (type: `integer`):

Cap on how many captures have their content retrieved across the whole run. Captures beyond this are still indexed, just without content.

## `jsRender` (type: `string`):

Some captures are an empty shell because the page built itself in the browser. Rendering recovers that text, and it runs the archived JavaScript from the same capture rather than today's code, so the result is period correct. Auto renders only captures that look like shells, which is the setting to use. Always renders every retrieved capture and is much slower.

## `generateReport` (type: `boolean`):

Save a readable summary to the key value store as REPORT.md, listing the changes ranked by how big they were, with pricing and title moves called out and an archive link on every row so a reader can check the claim. Use this when the output is for a person or a model rather than a pipeline.

## `monitor` (type: `boolean`):

Remember the newest capture seen for each URL and return only what has appeared since the previous run. Turn this on for a scheduled run so it reports new activity rather than the whole history every time.

## `timeoutSecs` (type: `integer`):

How long to wait for the archive before giving up on a single request. Large archived pages can be slow to serve.

## Actor input object example

```json
{
  "url": "bbc.co.uk/news",
  "useCase": "",
  "matchType": "exact",
  "from": "2024",
  "snapshotsPer": "none",
  "maxSnapshots": 100,
  "statusCodes": [
    "200"
  ],
  "mimeTypes": [
    "text/html"
  ],
  "detectChanges": false,
  "onlyChanged": false,
  "fetchContent": false,
  "includeRawHtml": true,
  "includeText": true,
  "maxTextChars": 0,
  "maxContentFetches": 25,
  "jsRender": "off",
  "generateReport": false,
  "monitor": false,
  "timeoutSecs": 30
}
```

# Actor output Schema

## `results` (type: `string`):

One row per item: archived captures with content.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "bbc.co.uk/news",
    "from": "2024",
    "statusCodes": [
        "200"
    ],
    "mimeTypes": [
        "text/html"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("spookyweb/wayback-machine-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "url": "bbc.co.uk/news",
    "from": "2024",
    "statusCodes": ["200"],
    "mimeTypes": ["text/html"],
}

# Run the Actor and wait for it to finish
run = client.actor("spookyweb/wayback-machine-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "bbc.co.uk/news",
  "from": "2024",
  "statusCodes": [
    "200"
  ],
  "mimeTypes": [
    "text/html"
  ]
}' |
apify call spookyweb/wayback-machine-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,spookyweb/wayback-machine-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Fs2o2jg9dri0nLwvI/builds/VBuRNSxbp9rjc5ZHN/openapi.json