Wayback Machine Scraper: Archive.org Snapshot History API

What does Wayback Machine Scraper do?

Wayback Machine scraper for the full capture history of any URL or domain via the official Internet Archive CDX API. One row per snapshot: capture date, HTTP status, MIME type, digest, contentChanged flag and a direct web.archive.org link. Ideal for SEO audits and change tracking.

📖 Step-by-step tutorial: https://dev.to/scrape-smith/how-to-get-the-full-wayback-machine-history-of-any-website-no-code-or-python-p5d

Popular searches this actor answers: Wayback Machine scraper, Internet Archive API, archive.org CDX API, website history, historical snapshots, page change detection.

Try it free: the input is prefilled, so just click Start and you get real results in seconds for a fraction of a cent.

What data does it return?

Real rows from a test run with the example input (truncated):

capturedAt	statusCode	mimeType	contentChanged	snapshotUrl
2020-01-09T13:07:05Z	200	text/html	None	https://web.archive.org/web/20200109130705/https://apify.com…
2020-02-24T14:06:03Z	200	text/html	True	https://web.archive.org/web/20200224140603/https://apify.com…
2020-03-02T11:10:47Z	200	text/html	True	https://web.archive.org/web/20200302111047/https://apify.com…
2020-04-05T10:34:45Z	200	text/html	True	https://web.archive.org/web/20200405103445/https://apify.com…
2020-05-01T11:22:48Z	200	text/html	True	https://web.archive.org/web/20200501112248/https://apify.com…

Full JSON for one row:

[
  {
    "input": "apify.com",
    "original": "https://apify.com/",
    "capturedAt": "2020-01-09T13:07:05Z",
    "timestamp": "20200109130705",
    "statusCode": "200",
    "mimeType": "text/html",
    "digest": "D4P2GPUK3PEGEFYQFUENG7MNTQI3ZRN4",
    "contentChanged": null,
    "sizeBytes": 29306,
    "snapshotUrl": "https://web.archive.org/web/20200109130705/https://apify.com/"
  }
]

Download results as JSON, CSV, Excel, XML or HTML, call it via the Apify API, or connect it to Zapier, Make, n8n, Google Sheets and AI agents (Apify MCP server).

Use cases

• SEO audits: see when titles, pages or redirects changed
• Competitor monitoring: track pricing-page and landing-page history
• Content recovery: find old versions of deleted pages
• Domain due diligence before buying an expired domain
• Academic and journalistic research on how sites evolved

How to use Wayback Machine Scraper

1. Open the actor and keep the prefilled example, or replace it with your own input.
2. Click Start and wait for the run to finish.
3. Export the dataset or fetch it via API.

Input

Field	Description
urls	URLs or domains to look up in the Wayback Machine, one per line. Bare domains (example.com) or full URLs (https://example.com/pricing). With matchType=domain a domain also covers its subdomains.
from	Earliest capture date to include, as YYYY or YYYYMMDD (e.g. 2020 or 20230115). Leave empty for the very first capture.
to	Latest capture date to include, as YYYY or YYYYMMDD. Leave empty for up to today.
collapse	How to thin out captures: month = at most one snapshot per month (good overview), day = one per day, digest = only when page content changed, none = every capture.
matchType	exact = only this URL, prefix = every URL starting with it (e.g. a whole /blog/ folder), domain = the whole domain including subdomains.
limit	Maximum snapshots returned per input URL (each snapshot is one billed row). Example: 200.

Example input:

{
  "urls": [
    "apify.com"
  ],
  "from": "2020",
  "collapse": "month",
  "limit": 20
}

Pricing

Pay per event: $0.003 per run start + $0.0005 per snapshot returned. 1,000 snapshots ≈ $0.50. Apify platform usage is included. Set a max charge per run to cap cost.

Example job	Approx. cost
Monthly history of one domain since 2020 (~80 snapshots)	≈ $0.04 + $0.003 start
1,000 snapshots	$0.50 + $0.003 start

FAQ

Is this the official Wayback Machine API?

It uses the Internet Archive's public CDX server API, the same index web.archive.org uses. It is not affiliated with the Internet Archive.

How do I only get snapshots where the page changed?

Set collapse to digest, or check the contentChanged flag on each row.

Does it download page HTML?

No. It returns snapshot metadata and a snapshotUrl you can open or fetch yourself.

Is it legal to use this data?

The actor only reads public, official endpoints. You are responsible for using the data in line with the source's terms and applicable laws (e.g. GDPR for personal data).

API example

curl -X POST "https://api.apify.com/v2/acts/scrape-smith~wayback-snapshots/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H 'Content-Type: application/json' -d '{"urls": ["apify.com"], "from": "2020", "collapse": "month", "limit": 20}'

More Scrape-Smith actors

• Shopify Products Scraper
• Apple App Store Reviews Scraper
• USPTO Trademark Status Lookup
• ATS Jobs Scraper
• Bluesky Scraper
• Apple Podcasts Scraper
• Steam Reviews Scraper