# Wayback Machine Snapshot History & Archived Text Extractor (`dataspryva/wayback-machine-snapshot-history-and-archived-text`) Actor

Pull the Wayback Machine's capture history for any URL, host, or domain with timestamps, status codes, MIME types, and optional archived page text.

- **URL**: https://apify.com/dataspryva/wayback-machine-snapshot-history-and-archived-text.md
- **Developed by:** [Dataspryva](https://apify.com/dataspryva) (community)
- **Categories:** SEO tools, Automation, Developer tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.49 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Wayback Machine Snapshot History & Archived Text Extractor

Turn the Internet Archive's public Wayback Machine into a structured dataset. Give this actor one URL, a host, an entire domain, or a bulk list, then choose a capture history, a version timeline, an archive-coverage summary, the closest snapshot to a date, or the readable text of the archived page itself.

No account, API key, cookie, or proxy setup is needed. The actor talks to the same public CDX and availability endpoints that power the Wayback Machine website.

### Modes

| Mode | What you get | Typical use |
|---|---|---|
| `search` | One row per archived capture with timestamp, playback URL, HTTP status, MIME type, digest, and size | Full capture history, broken-link recovery, redirect mapping |
| `timeline` | One row per distinct content version, with first/last seen dates, capture count, and duration | See how a page changed without reading every duplicate capture |
| `summary` | One archive-coverage row per target: scanned capture count, version count, first/last capture, days covered, status and year breakdowns | Vet expired domains, measure archive density, compare sites |
| `closest` | The capture nearest to a target date, with its playback URL and distance in days | Legal, compliance, and journalism checks: "what did the page say on this date?" |
| `content` | Readable text plus title, description, language, canonical URL, headings, word count, character count, and text hash from archived HTML | Recover deleted copy, search archived text, build text corpora |

### Bulk targets and pagination

- Put one target in `url` or many in `urls`; both are merged and deduplicated.
- The `maxItems` budget is split evenly across targets so every target gets a fair share.
- Capture history uses automatic resume-key pagination, so a single run can retrieve up to 100,000 captures instead of stopping at the first CDX page.
- `sortDirection` chooses whether the limited result set comes from the oldest or newest end of the archive. Search rows are chronological; `latest` reverses the final rows so the newest capture appears first.
- `collapseBy` applies to `search` mode: keep every raw capture, one per distinct content digest, one per URL, or one per month/day/hour. `timeline` and `summary` always read raw captures so version durations and coverage counts are not flattened.
- `summary` scans up to `maxItems` captures per target, split between the oldest and newest ends, so the first and last capture stay accurate even on dense domains. `captureCount`, `versionCount`, and the breakdowns describe that scanned window; raise `maxItems` for a fuller census.

### Output fields

#### Capture rows (`search`)

`originalUrl`, `urlKey`, `timestamp`, `archiveDate`, `archiveUrl`, `rawArchiveUrl`, `sourceUrl`, `statusCode`, `statusText`, `statusClass`, `mimeType`, `contentDigest`, `contentLength`, `host`, `urlPath`, `fileExtension`, `captureYear`, `matchType`, `isRevisit`, `recordType`, `resultIndex`, `scrapedAt`.

With `detectChanges` on, each row also gets `captureIndex`, `changed`, `changeType` (`initial`, `content`, `status`, `contentAndStatus`, `unchanged`), `isInitialCapture`, `previousTimestamp`, `previousStatusCode`, `previousContentDigest`, and `daysSincePrevious`.

With `enrichWithContent` on, rows also get the archived-text fields below, up to `maxContentFetch` rows.

#### Version rows (`timeline`)

`originalUrl`, `firstTimestamp`, `lastTimestamp`, `firstArchiveDate`, `lastArchiveDate`, `captureCount`, `durationDays`, `archiveUrl`, `rawArchiveUrl`, `contentDigest`, `mimeType`, `statusCode`, `statusText`, `contentLength`, `sourceUrl`, `recordType`, `resultIndex`, `scrapedAt`.

#### Summary rows (`summary`)

`originalUrl`, `matchType`, `archived`, `captureCount`, `versionCount`, `firstTimestamp`, `lastTimestamp`, `firstArchiveDate`, `lastArchiveDate`, `firstArchiveUrl`, `lastArchiveUrl`, `spanDays`, `statusBreakdown`, `mimeTypeBreakdown`, `activeYears`, `yearBreakdown`, `sourceUrl`, `recordType`, `resultIndex`, `scrapedAt`.

#### Closest-snapshot rows (`closest`)

`originalUrl`, `available`, `requestedTimestamp`, `requestedDate`, `timestamp`, `archiveDate`, `archiveUrl`, `rawArchiveUrl`, `statusCode`, `statusClass`, `statusText`, `captureDistanceDays`, `sourceUrl`, `recordType`, `resultIndex`, `scrapedAt`.

#### Archived-page rows (`content`)

`originalUrl`, `timestamp`, `archiveDate`, `archiveUrl`, `rawArchiveUrl`, `statusCode`, `mimeType`, `contentDigest`, `contentLength`, `title`, `description`, `language`, `canonicalUrl`, `metaKeywords`, `headings`, `text`, `wordCount`, `characterCount`, `textHash`, `contentTruncated`, `fetchContentLength`, `fetchContentTruncated`, `contentMimeType`, `sourceUrl`, `recordType`, `resultIndex`, `scrapedAt`.

Empty values are omitted at every depth, so a field appears only when the archive actually supplied it.

### Input

| Field | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `search` | `search`, `timeline`, `summary`, `closest`, or `content` |
| `url` | string | – | Single URL, host, domain, or playback URL |
| `urls` | array | `[]` | Bulk targets; merged with `url` |
| `matchType` | string | `exact` | `exact`, `prefix`, `host`, or `domain`; wildcards are auto-detected |
| `dateFrom` | string | – | Earliest capture date, inclusive |
| `dateTo` | string | – | Latest capture date, inclusive |
| `statusCodes` | array | `[]` | Keep only these HTTP status codes |
| `excludeStatusCodes` | array | `[]` | Drop these HTTP status codes |
| `mimeTypes` | array | `[]` | Keep only these MIME types |
| `excludeMimeTypes` | array | `[]` | Drop these MIME types |
| `collapseBy` | string | `digest` | `none`, `digest`, `urlkey`, `monthly`, `daily`, or `hourly`; applies to `search` only |
| `sortDirection` | string | `earliest` | `earliest` or `latest` |
| `targetDate` | string | – | Reference date for `closest` and `content`; defaults to now |
| `enrichWithContent` | boolean | `false` | Add archived text to search rows |
| `maxContentFetch` | integer | `10` | Text-fetch cap for search enrichment (0–500) |
| `maxContentChars` | integer | `500000` | Text cap per archived page (1,000–2,000,000) |
| `includeHeadings` | boolean | `true` | Include H1–H6 headings in archived text |
| `maxItems` | integer | `500` | Record cap (1–100,000) |
| `detectChanges` | boolean | `true` | Annotate content/status changes in search rows |

The daily-test prefill is the cheapest possible run: `mode=closest`, `url=apify.com`, `targetDate=2024-01-01`, `maxItems=1`. It performs one availability lookup and returns one snapshot record.

#### Example: full capture history with changes

```json
{
    "mode": "search",
    "url": "https://example.com/",
    "matchType": "exact",
    "dateFrom": "2020-01-01",
    "dateTo": "2024-12-31",
    "statusCodes": ["200"],
    "collapseBy": "none",
    "detectChanges": true,
    "maxItems": 500
}
```

#### Example: bulk closest snapshots

```json
{
    "mode": "closest",
    "urls": ["apify.com", "example.com", "wikipedia.org"],
    "targetDate": "2020-06-15",
    "maxItems": 3
}
```

#### Example: archived text from a playback URL

```json
{
    "mode": "content",
    "url": "https://web.archive.org/web/20240101000000/https://apify.com/",
    "maxItems": 1,
    "includeHeadings": true
}
```

### Legacy inputs

Saved tasks and integrations from earlier versions keep working. The actor still accepts `url` as a single target. Old field names `maxResults`, `includeContent`, `statusFilter`, and `mimeFilter` are mapped onto `maxItems`, `enrichWithContent`, `statusCodes`, and `mimeTypes` in code. Collapse aliases `timestamp:6`, `timestamp:8`, and `timestamp:10` are accepted alongside `monthly`, `daily`, and `hourly`.

### Use cases

- **Content recovery** — pull the text of pages that no longer exist
- **SEO link-rot audits** — list archived 404s, redirects, and deleted pages on a domain
- **Competitive history** — compare pricing, messaging, or product pages month by month
- **Legal and journalism evidence** — pin the closest capture to a specific date
- **Domain due diligence** — measure archive age, density, gaps, and version count before buying an expired domain
- **Dataset building** — export archived text and metadata for search, NLP, or research

### FAQ

**Where does the data come from?**
The Internet Archive's public Wayback CDX search API, its availability API, and archived playback pages. No browser UI is scraped.

**Do I need an Internet Archive account or API key?**
No. Every request is anonymous and cookieless.

**Can I search the archive by keyword?**
The CDX API indexes URLs and capture metadata, not full text. Use `mode=content` to download page text and filter it in your own pipeline.

**Why are some captures marked `warc/revisit`?**
The archive sometimes records a revisit instead of a full copy. Those rows keep the digest and size but may not have an HTTP status code.

**Why is `content` missing from a row?**
The row was beyond the text-fetch budget, the archived resource was not HTML/text, or replay returned an error. `contentTruncated` is set when text was clipped at `maxContentChars`.

**Why does `closest` mode omit MIME type and digest?**
The availability endpoint returns the nearest capture URL, timestamp, and status only. Use `search` mode with a narrow date range when you need full CDX metadata.

**Can I retrieve more than 10,000 captures?**
Yes. The actor pages through the CDX index with resume keys, so `maxItems` can go up to 100,000. Very broad host/domain queries and `collapseBy=urlkey` can be slower on the archive side.

**Is archive coverage complete?**
No. The Wayback Machine only stores what its crawlers captured. Popular pages usually have dense history; obscure pages may have none, and a summary row with `captureCount=0` is still a valid result.

**Is this affiliated with the Internet Archive?**
No. It is an independent tool that reads the Internet Archive's public Wayback Machine endpoints.

# Actor input Schema

## `mode` (type: `string`):

What to collect. 'search' returns every matching capture; 'timeline' returns one row per distinct content version; 'summary' returns archive coverage per target; 'closest' returns the nearest capture to a date; 'content' returns readable text from the archived page.

## `url` (type: `string`):

One URL, host, domain, or Wayback playback URL. Examples: 'apify.com', 'https://example.com/blog', 'https://web.archive.org/web/20240101000000/https://apify.com/'. Wildcards like 'example.com/*' and '*.example.com' are detected automatically.

## `urls` (type: `array`):

One or more targets for batch collection. Each target is queried independently and the Max items budget is split across them.

## `matchType` (type: `string`):

How the target is matched against the archive. 'exact' = that URL only; 'prefix' = the URL and every path beneath it; 'host' = every path on the same host; 'domain' = the host and all subdomains.

## `dateFrom` (type: `string`):

Earliest capture date, inclusive. Accepts YYYY, YYYY-MM, YYYY-MM-DD, or a full timestamp such as 2024-01-31T12:00:00.

## `dateTo` (type: `string`):

Latest capture date, inclusive. Uses the same formats as From date. A date without a time includes the whole day.

## `statusCodes` (type: `array`):

Only keep captures with these HTTP status codes, e.g. \['200','301']. Leave empty for all statuses. Values may also be comma-separated.

## `excludeStatusCodes` (type: `array`):

Drop captures with these HTTP status codes, e.g. \['404','500']. Leave empty to exclude nothing.

## `mimeTypes` (type: `array`):

Only keep captures with these MIME types, e.g. \['text/html','application/pdf']. Leave empty for all MIME types.

## `excludeMimeTypes` (type: `array`):

Drop captures with these MIME types, e.g. \['image/png','video/mp4']. Leave empty to exclude nothing.

## `collapseBy` (type: `string`):

Deduplicate adjacent captures in 'search' mode. 'digest' keeps one record per distinct content fingerprint; 'urlkey' keeps one capture per URL; 'monthly', 'daily', and 'hourly' keep one capture per time bucket; 'none' returns every raw capture. 'timeline' and 'summary' always read raw captures so their counts and durations stay accurate.

## `sortDirection` (type: `string`):

Choose whether the limited result set comes from the oldest or newest captures. Search rows are emitted in chronological order; 'latest' reverses the final rows so the newest appears first.

## `targetDate` (type: `string`):

Date used by 'closest' and 'content' modes, and as the reference when a playback URL does not include a timestamp. Accepts YYYY, YYYY-MM, YYYY-MM-DD, or a full timestamp. Defaults to now when empty.

## `enrichWithContent` (type: `boolean`):

In 'search' mode, fetch the archived HTML for the first Max content fetches rows and add its readable text and metadata. Off by default because it costs extra requests.

## `maxContentFetch` (type: `integer`):

Cap on archived pages downloaded when 'Add archived text to search rows' is on. 0 disables enrichment. This setting is ignored by 'content' mode, which uses Max items instead.

## `maxContentChars` (type: `integer`):

Truncate extracted text at this many characters. The full word count and text hash are still recorded, and contentTruncated is set when clipping happens.

## `includeHeadings` (type: `boolean`):

Add the page's H1-H6 headings as structured records when archived text is extracted.

## `maxItems` (type: `integer`):

Hard cap on emitted records. For bulk search the budget is split evenly across targets. In 'summary' mode it is also the per-target capture scan budget, split between the oldest and newest ends. Up to 100,000 captures are supported through automatic resume-key pagination.

## `detectChanges` (type: `boolean`):

In 'search' mode, compare each capture with the previous capture of the same URL and add changed, changeType, previousTimestamp, and related fields.

## Actor input object example

```json
{
  "mode": "closest",
  "url": "apify.com",
  "urls": [],
  "matchType": "exact",
  "statusCodes": [],
  "excludeStatusCodes": [],
  "mimeTypes": [],
  "excludeMimeTypes": [],
  "collapseBy": "none",
  "sortDirection": "earliest",
  "targetDate": "2024-01-01",
  "enrichWithContent": false,
  "maxContentFetch": 0,
  "maxContentChars": 500000,
  "includeHeadings": false,
  "maxItems": 1,
  "detectChanges": false
}
```

# Actor output Schema

## `records` (type: `string`):

Dataset containing snapshot, version, summary, closest-snapshot, archived-page, or diagnostic records for the selected mode.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "mode": "closest",
    "url": "apify.com",
    "urls": [],
    "matchType": "exact",
    "statusCodes": [],
    "excludeStatusCodes": [],
    "mimeTypes": [],
    "excludeMimeTypes": [],
    "collapseBy": "none",
    "sortDirection": "earliest",
    "targetDate": "2024-01-01",
    "enrichWithContent": false,
    "maxContentFetch": 0,
    "maxContentChars": 500000,
    "includeHeadings": false,
    "maxItems": 1,
    "detectChanges": false
};

// Run the Actor and wait for it to finish
const run = await client.actor("dataspryva/wayback-machine-snapshot-history-and-archived-text").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "mode": "closest",
    "url": "apify.com",
    "urls": [],
    "matchType": "exact",
    "statusCodes": [],
    "excludeStatusCodes": [],
    "mimeTypes": [],
    "excludeMimeTypes": [],
    "collapseBy": "none",
    "sortDirection": "earliest",
    "targetDate": "2024-01-01",
    "enrichWithContent": False,
    "maxContentFetch": 0,
    "maxContentChars": 500000,
    "includeHeadings": False,
    "maxItems": 1,
    "detectChanges": False,
}

# Run the Actor and wait for it to finish
run = client.actor("dataspryva/wayback-machine-snapshot-history-and-archived-text").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "mode": "closest",
  "url": "apify.com",
  "urls": [],
  "matchType": "exact",
  "statusCodes": [],
  "excludeStatusCodes": [],
  "mimeTypes": [],
  "excludeMimeTypes": [],
  "collapseBy": "none",
  "sortDirection": "earliest",
  "targetDate": "2024-01-01",
  "enrichWithContent": false,
  "maxContentFetch": 0,
  "maxContentChars": 500000,
  "includeHeadings": false,
  "maxItems": 1,
  "detectChanges": false
}' |
apify call dataspryva/wayback-machine-snapshot-history-and-archived-text --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,dataspryva/wayback-machine-snapshot-history-and-archived-text"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/mXENdRBun6FUTc2pI/builds/IADcjvMrlQA9u17xO/openapi.json