🕰️ Wayback Machine Scraper: Snapshots & Website History

Wayback Machine Scraper exports archived snapshots from web.archive.org for any URL or domain: capture date, HTTP status, MIME type, size, content digest and a direct archive link per row. It returns one row per content change, lists every archived URL of a domain with capture counts, checks the closest snapshot to any date, searches archive.org books, video, audio and software, and can return only what is new since your last run. Export CSV, Excel, JSON or XML. No login, no API key. Priced from $15 per 1,000 snapshots and $1 per 1,000 archive.org items on the free tier.

Wayback Machine Scraper reads the Internet Archive's public CDX, availability and advanced search APIs and returns one clean row per capture, URL or item. For example, nytimes.com returns its first capture from 1996-11-12 and pages through 1,000 captures per request, so histories are complete. Built for SEO teams, lawyers, journalists, researchers and domain buyers.

🔎 What does Wayback Machine Scraper do?

• Website history by URL: every archived capture of a page, with ISO timestamp, status code, MIME type and urlKey.
• One row per content change: a new row appears only when the page digest differs from the previous capture. Only adjacent identical digests collapse, so a page that goes A, B, A returns 3 rows.
• All archived URLs of a domain: match type domain plus one per unique URL inventories a site, subdomains included; inventory statistics add captureCount, firstSeen and lastSeen.
• Filters and timelines: HTTP status (404, 200,301,302, 3xx, !200), MIME type with ! to exclude, and one capture per day, month or year.
• Availability check: batch check, closest snapshot to any date (YYYYMMDD).
• Archive.org item search: books, video, audio, software and images by text or field query such as creator:"Mark Twain".
• Only new results: schedule the actor and get just the rows no earlier run delivered. Skipped rows are not charged.

📋 What data can you extract from the Wayback Machine?

Each record type fills only the keys it has (44 keys in the dataset schema).

Field	Description
recordType	snapshot, availability or item
target	The URL, domain or query you asked about
originalUrl	URL as it was archived
snapshotAt	Capture time, ISO 8601 (UTC)
timestamp	Raw Wayback timestamp (YYYYMMDDhhmmss), plus urlKey
statusCode	HTTP status at capture time (when recorded)
mimeType	Content type (when recorded)
sizeBytes	Archived size in bytes
digest	Content hash, equal hashes mean unchanged content
waybackUrl	Direct web.archive.org link to the capture
captureCount, firstSeen, lastSeen	Inventory mode: captures of the URL and first and last capture time
isArchived, closestSnapshotAt, requestedTimestamp	Availability rows: archived or not, nearest capture, date asked
itemId, title, creator, description, mediaType, itemDate, year, language, publisher, subjects, collections, formats	Item rows: archive.org identity and descriptive data
downloads, downloadsLastWeek, downloadsLastMonth, itemSizeBytes, fileCount, pageCount, avgRating, reviewCount, licenseUrl, publishedAt, addedAt, itemUrl, thumbnailUrl	Item rows: counts, size, rating, license, dates, links
scrapedAt	When the row was collected (UTC)

The Store output has 5 views: Snapshots, URL inventory, Content versions, Availability and Archive.org items. Personal fav-* collections are filtered out.

📊 Sample output of the website history export

Real row from a run on nytimes.com:

{
  "recordType": "snapshot",
  "target": "nytimes.com",
  "originalUrl": "http://www.nytimes.com:80/",
  "snapshotAt": "1996-11-12T18:15:13Z",
  "timestamp": "19961112181513",
  "statusCode": 200,
  "mimeType": "text/html",
  "digest": "GY3YVZK6NIR7GKGXGGK4GPS2ZORULYDB",
  "waybackUrl": "https://web.archive.org/web/19961112181513/http://www.nytimes.com:80/"
}

Real availability row for apify.com with target date 20200101:

{
  "recordType": "availability",
  "target": "apify.com",
  "isArchived": true,
  "closestSnapshotAt": "2019-12-30T05:20:25Z",
  "statusCode": 200,
  "requestedTimestamp": "20200101"
}

💰 How much does it cost to scrape the Wayback Machine?

Pay per event. You pay only for rows that were delivered.

Event	Free tier price	When it is charged
snapshot-record	$0.015 (so $15 per 1,000)	One archived capture, content version or inventory URL saved to the dataset
availability-record	$0.010	One URL that has an archived snapshot
item-record	$0.001 (so $1 per 1,000)	One archive.org item row saved to the dataset

Worked example at the Free tier: 1,000 snapshot rows cost 1,000 x $0.015 = $15.00, 1,000 availability checks that all find a snapshot cost 1,000 x $0.010 = $10.00, and 1,000 archive.org item rows cost 1,000 x $0.001 = $1.00. Digest, status-filter and inventory rows bill as snapshot-record.

Paid plans are cheaper (Gold is $10.09 per 1,000 snapshots; item rows are $0.001 on every tier). Failed requests, errors, "never archived" answers, empty searches and rows skipped in only-new mode are not charged. A maximum cost per run stops the actor cleanly. Free users get a 10-row preview.

🚀 How to scrape website history in 3 steps

1. Open the actor and click Try for free.
2. Add URLs or domains, pick a match type, and optionally set dates, a dedupe mode or filters.
3. Click Start, then download CSV, Excel, JSON or XML.

⚙️ Input

Field	Meaning
urls	Pages or domains for snapshot history
matchType	exact, prefix, host or domain (host plus subdomains)
fromDate / toDate	Range as YYYY, YYYYMM or YYYYMMDD
onlySuccessful	Keep only HTTP 200 captures (ignored when statusCodes is set)
statusCodes	404, 200,301,302, 3xx, !200, or - for revisit records
mimeFilter	For example application/pdf; start with ! to exclude
collapse	none, daily, monthly, yearly, digest, unique-urls
inventoryStats	With unique-urls: add captureCount, firstSeen, lastSeen
maxScanRows	Captures read for statistics and only-new runs (default 200,000)
checkAvailabilityUrls	URLs for availability checks
availabilityTimestamp	Target date for availability, YYYYMMDD
itemSearchQueries	archive.org text or field queries, one per line
itemMediaType	any, texts, movies, audio, software, image and more
itemSortBy	relevance, downloads, newestAdded, publicationDate, title and more
onlyNew / stateKey / resetSeen	Monitor mode, optional monitor name, clear previous results
maxItems	Cap on rows returned and paid

{
  "urls": ["apify.com"],
  "matchType": "domain",
  "collapse": "digest",
  "statusCodes": "200",
  "maxItems": 10
}

Invalid dates, a bad statusCodes value or an input with no URLs or queries fail immediately and cost nothing.

📦 Output

One dataset row per snapshot, URL, availability check or archive.org item, exportable as JSON, CSV, Excel or XML. An empty search finishes as succeeded with zero charges. If Wayback Machine is down or blocks the request and no rows were delivered, the run fails.

Limits to know:

• Inventory statistics read every capture, so they are slower. A URL with more captures than maxScanRows, or than can be read in 120 seconds (about 100,000 to 150,000), ends the run as failed with an explanation and no charge. Root pages of huge domains such as wikipedia.org hit this: use matchType exact or prefix, or narrow the dates.
• Item search stops at 10,000 results per query; only-new memory keeps the newest 200,000 ids per monitor.

⚖️ Wayback Machine scraper vs alternatives

Store pages read on 2026-10-10 (Free tier prices):

Feature	This actor	parseforge	rowfeed
Price per 1,000 rows	$15	from $8.25 (plus start fee)	from $0.50
Status and MIME filters	Both, with exclusion	Both, with invert	Status only
One row per content change	Yes	Yes	Yes
Inventory counts, first and last seen	Yes	Yes	Not shown
Availability with target date	Yes	Not shown	Latest only
Archive.org item search	Yes	Not shown	Yes
Only-new monitoring	Yes	Not shown	Not shown

We are the most expensive per row in this group. parseforge lists $0.011 per result plus a $0.005 start charge in its README. benthepythondev (from $1.20) and smorgi_apps (from $2.00) offer digest collapse and basic filters only, and neither page shows availability checks or item search. If you only need a plain, cheap list of captures, a cheaper actor fits. The extra price buys one actor for history, content versions, inventory counts, dated availability, item search and only-new schedules, with error and empty rows never charged.

💼 Use cases

SEO recovery of expired or migrated domains, legal evidence with timestamped capture lists, change tracking in only-new mode, domain due diligence, and finding books, films and software on archive.org ranked by downloads.

🔌 Run via API, schedule and integrations

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });
const run = await client.actor('RecordsData/wayback-machine-scraper').call({
  urls: ['example.com'], collapse: 'digest', statusCodes: '200', maxItems: 100,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();

Schedule runs and send results to Zapier, Make, n8n, Google Sheets or Slack.

🤖 Wayback Machine Scraper MCP server for AI agents

AI agents can call this Actor through the Apify MCP server at https://mcp.apify.com?tools=RecordsData/wayback-machine-scraper. The URL loads only this Actor as a tool.

Add this to your MCP client configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=RecordsData/wayback-machine-scraper"
    }
  }
}

Example agent prompt:

Use the Wayback Machine Scraper to get one row per content change of apify.com with matchType exact, collapse digest and statusCodes 200, then tell me the earliest capture date and when the page content changed most recently.

Tool description: choose this Actor when an agent needs the archive history of a URL or domain, one row per content change, an inventory of archived URLs with capture counts, a batch check of whether URLs are archived, or a search of archive.org items. It takes the same fields as the Input table.

🛡️ Is it legal to scrape the Wayback Machine?

The actor uses the Internet Archive's public CDX, availability and search APIs at a polite rate with an identified user agent. It returns metadata and links, not archived page content. Respect archive.org's terms and the copyright of archived pages when you reuse content.

❓ Frequently asked questions

How far back do Wayback Machine snapshots go?

The Wayback Machine Scraper returns the exact capture time of every snapshot, so depth depends on the site. In our test, nytimes.com goes back to 1996-11-12. Each row has snapshotAt in ISO 8601 (UTC), the raw timestamp, status code and MIME type when recorded, and a direct web.archive.org link. Use fromDate and toDate to limit the range.

Why did my Wayback Machine Scraper run return 0 results?

Either nothing was archived for that URL and match type, or your filters are too narrow. Try matchType prefix or domain, remove the date, status and MIME filters, and check the URL for typos. In only-new mode, zero rows also means nothing changed since the last run. An empty search finishes as succeeded and costs nothing.

What is the difference between the Wayback Machine Scraper match types?

In the Wayback Machine Scraper, exact returns snapshots of one page, prefix everything under a path, host one hostname, and domain the host plus all its subdomains. Choose exact for a single page history, and domain with the unique-urls dedupe mode to inventory a whole website.

How do I list every URL archived for a domain on the Wayback Machine?

Use the Wayback Machine Scraper with matchType set to domain and collapse set to unique-urls. That returns one row per distinct archived URL for the domain, including subdomains, each with a capture time and a direct web.archive.org link. Turn on inventoryStats to add captureCount, firstSeen and lastSeen, and add statusCodes 200 to keep only pages that returned HTTP 200.

Can the Wayback Machine Scraper find only archived PDFs?

Yes. Set the mimeFilter input to application/pdf and use matchType prefix or domain with your site. The output then contains only archived PDF captures, each with capture time, size in bytes, content digest and direct web.archive.org link. The same filter works for other MIME types, and a leading ! excludes a type.

Do I pay for URLs that were never archived on the Wayback Machine?

No. When the Wayback Machine Scraper checks a URL that was never archived, it returns an availability row with isArchived set to false, and that row is not charged. The availability event of $0.010 on the Free tier applies only to URLs that have an archived snapshot. Failed requests, errors and empty searches are also never charged.

Does the Wayback Machine Scraper download the archived page content?

No. The Wayback Machine Scraper returns metadata and links only: capture time, status code, MIME type, size, content digest and the waybackUrl link. It uses the Internet Archive's CDX and availability APIs and does not fetch the archived pages. Equal digest values across captures tell you the content was unchanged.

Can I try the Wayback Machine Scraper for free?

Yes. Free Apify users get a 10-row preview of the Wayback Machine Scraper, enough to check snapshot dates, status codes and archive links for your own URL. Paid plans unlock full runs and lower prices, with Gold at $10.09 per 1,000 snapshots instead of $15.

How do I get one row per content change on the Wayback Machine?

Set collapse to digest in the Wayback Machine Scraper. The CDX server then returns a new row only when the content digest differs from the previous capture, so a long history shrinks to its real versions. It compares adjacent captures only, so A, B, A returns 3 rows. Combine it with statusCodes 200 to skip error captures.

Does the Wayback Machine Scraper search archive.org books and videos?

Yes. Fill itemSearchQueries with a text or field query such as creator:"Mark Twain", pick an item type and a sort order, and the Wayback Machine Scraper returns one row per item with title, creator, downloads, formats and the item link. Item rows cost $0.001 each ($1 per 1,000), and each query stops at 10,000 results.

How do I get only new Wayback Machine results on a schedule?

Turn on onlyNew in the Wayback Machine Scraper and schedule the run. Rows that an earlier run with the same inputs already delivered are skipped and not charged, so a run with nothing new returns zero rows and costs nothing. Use stateKey to name separate monitors and resetSeen to start again from zero.

🔗 Want more data? Other RecordsData scrapers

• Wikipedia Scraper: Articles, Summaries & Pageviews
• Hacker News Scraper: Story & Comment Search
• USPTO Trademark Status Scraper

💬 Support

Found a bug or a missing field? Open the Issues tab or write to contact.punkrecordsdata@gmail.com.

Last updated: 2026-10-11