Wayback Machine Scraper

Search the Internet Archive's Wayback Machine for historical snapshots of any URL or domain. Uses the CDX API for batch snapshot retrieval with Availability API fallback for the single closest snapshot. Returns clean typed JSON — no API key required.

Why this actor?

• CDX + Availability dual engine — fast batch retrieval via CDX, with graceful fallback to the Availability API for edge cases
• Per-snapshot billing — pay only for the snapshots you actually receive ($0.003 each), not per-request
• Full metadata — timestamp, original URL, HTTP status code, MIME type, content digest, and length for every snapshot
• Direct archive links — every snapshot row includes a clickable https://web.archive.org/web/TIMESTAMP/URL link
• Date filtering — restrict to specific date ranges with from/to in YYYYMMDD format
• No auth, open service — the Wayback Machine is entirely public

Use cases

• Change detection — monitor how a website's content has evolved over time
• Historical research — retrieve archived versions of defunct or changed pages
• Compliance and legal — produce timestamped evidence of web content at specific dates
• Content recovery — find cached copies of pages that are no longer live
• SEO analysis — audit historical versions of competitor pages

Input

Field	Type	Default	Description
urls	string[]	(required)	One or more URLs or domains to search. Use *.example.com for domain wildcards. Capped at 1000, deduplicated.
from	string	—	Earliest snapshot date in YYYYMMDD format (e.g. 20200101).
to	string	—	Latest snapshot date in YYYYMMDD format (e.g. 20231231).
limit	integer	100	Max snapshots per URL (1–1000). This is a billing boundary.
includeAvailability	boolean	true	Fall back to Availability API when CDX returns no snapshots.

Example

{
  "urls": ["*.github.com", "example.com"],
  "from": "20230101",
  "to": "20231231",
  "limit": 50
}

Output

Each dataset row is a flat snapshot object:

{
  "input": "github.com",
  "timestamp": "20230615123456",
  "originalUrl": "https://github.com/",
  "statuscode": "200",
  "mimetype": "text/html",
  "digest": "ABC123DEF456...",
  "length": "54321",
  "snapshotUrl": "https://web.archive.org/web/20230615123456/https://github.com/",
  "error": null,
  "scraped_at": "2026-08-03T12:00:00.000Z"
}

Error items carry a non-null error field and are never charged.

Pricing

Pay per event, and the platform usage is on us — the price you see is the price you pay, with no compute bill on top.

Event	Price	What one charge buys
Wayback Machine snapshot	$0.003	Charged for each Wayback Machine snapshot returned. Empty results are never charged. Errors and parse failures are free.
Actor Start	$0.002	One run, whatever it returns.

Higher Apify subscription tiers pay less on every event (Silver −20%, Gold −35%).

Reliability

• <2% failure rate target over 30 days
• Dual-API architecture: CDX fails over to Availability API automatically
• All upstream parsing logic is isolated in src/source.ts for fast repair
• Wayback Machine has been operational since 2001 with no authentication required

Limitations

• Not a real-time scraper — Wayback Machine snapshots may lag by hours to days
• Snapshot content is not extracted (links and metadata only)
• CDX API may return large result sets on busy domains; use from/to and limit to control volume

FAQ

What's the difference between CDX and Availability? CDX returns ALL snapshots matching your query (great for batch retrieval). The Availability API returns the single closest snapshot (useful as a fallback when CDX returns nothing).

How do I search an entire domain? Prefix with *. — e.g. *.example.com returns all snapshots from any path on example.com.

Are error items charged? No. Items with a non-null error field are always free.

---

This Actor is an independent tool and is not affiliated with the Internet Archive.