Wayback Machine Bulk Lookup

Look up Wayback Machine (archive.org) snapshots for any URL or list of URLs. Returns the full capture timeline, optional snapshot HTML-to-markdown content, and a live-vs-snapshot text diff. Built for OSINT analysts, journalists verifying sources, SEO teams recovering link-rot, and legal evidence collection.

---

What this actor does

For each input URL, the actor:

1. Queries the Wayback CDX API to retrieve the snapshot index in your specified date range and capture limit
2. Optionally fetches snapshot HTML for each capture and converts it to markdown (for reading or archiving)
3. Optionally fetches the current live URL and computes a line-level text diff against the most recent snapshot (to detect page changes)

Each output record contains the full snapshot timeline plus optional diff and content fields.

---

Input

Field	Type	Default	Description
urls	array of strings	—	Required. URLs to look up, with or without https:// or www.. Each is looked up as you wrote it and echoed back in url, so results join straight to your list
maxItems	integer	100	Maximum number of URLs to look up (one record per URL, in input order). 0 means no limit
dateFrom	string	—	Earliest snapshot date to include (ISO date, e.g. 2020-01-01), from the start of that day
dateTo	string	—	Latest snapshot date to include (ISO date, e.g. 2024-12-31), through the end of that day
captureLimit	integer	100	Max snapshots per URL (the oldest in the range). hasMoreSnapshots tells you when more exist
fetchSnapshotContent	boolean	false	Download snapshot HTML and convert to markdown
diffWithLive	boolean	false	Compute text diff between latest snapshot and current live URL

Example input:

{
  "urls": [
    "https://example.com/news/2024-article",
    "https://example.com/about"
  ],
  "dateFrom": "2023-01-01",
  "dateTo": "2024-12-31",
  "captureLimit": 50,
  "diffWithLive": true
}

---

Output

One record per input URL, always. Only captures the archive recorded with HTTP status 200 are listed (redirect and error captures are not). A URL the archive has never captured is success with snapshotCount 0; a lookup that could not be completed is error or timeout with errorMsg set and snapshotCount null, never a zero.

Field	Type	Description
url	string	The input URL exactly as you gave it
queriedUrl	string	The URL that was looked up (your input without any trailing annotation)
snapshotCount	number	Number of snapshots returned (at most captureLimit); null when the lookup failed
hasMoreSnapshots	boolean	true when the URL has more snapshots in the range than captureLimit
firstCaptured	string	Oldest returned snapshot (ISO 8601)
lastCaptured	string	Most recent snapshot in the range (ISO 8601), even when the list was truncated
captures	array	Snapshot entries — each a JSON-encoded string with timestamp, archiveUrl, status, mimetype, digest (identical digests mean identical content), length, and optionally contentMarkdown
diff	object	{ addedLines, removedLines, changedRatio } — null unless diffWithLive=true
liveStatus	number	Current HTTP status of the live URL — null unless diffWithLive=true and the site answered
finalLiveUrl	string	Final URL after redirects — null unless diffWithLive=true
status	string	success, partial (snapshots found but an optional step such as the live diff failed, see errorMsg), timeout, or error
errorMsg	string	Why the lookup or an optional step failed, null on success

Example output record:

{
  "url": "https://example.com/news/2024-article",
  "queriedUrl": "https://example.com/news/2024-article",
  "snapshotCount": 14,
  "hasMoreSnapshots": false,
  "firstCaptured": "2024-03-12T08:42:00Z",
  "lastCaptured": "2026-04-29T22:11:00Z",
  "captures": [
    "{\"timestamp\":\"2026-04-29T22:11:00Z\",\"archiveUrl\":\"https://web.archive.org/web/20260429221100/https://example.com/news/2024-article\",\"status\":200,\"mimetype\":\"text/html\",\"digest\":\"HT2DYGA5UKZCPBSFVCV3JOBXGW2G5UUA\",\"length\":1792}"
  ],
  "diff": { "addedLines": 12, "removedLines": 3, "changedRatio": 0.04 },
  "liveStatus": 200,
  "finalLiveUrl": "https://example.com/news/2024-article",
  "status": "success",
  "errorMsg": null
}

---

Dataset views

The actor produces two dataset views in the Apify console:

• Capture Timeline — url, snapshotCount, firstCaptured, lastCaptured, captures
• Live vs Snapshot Diff — url, liveStatus, diff, lastCaptured

---

Speed and large lists

The Wayback Machine is slow and rate-limited, so lists are looked up a few at a time. Expect roughly 5 to 10 URLs per minute; a list of a few hundred URLs finishes within the run, and the run timeout is 6 hours. A URL that cannot be answered after repeated attempts is returned as an error or timeout record and the rest of the list carries on. Heavily archived URLs that exceed captureLimit need one extra lookup to report the true latest capture. Turning on fetchSnapshotContent makes runs much longer: start with a small captureLimit (e.g. 10).

---

Use cases

• OSINT / research: Check whether a source URL existed, when it was captured, and how its content has changed
• Journalism: Verify archived versions of articles or government pages for fact-checking
• SEO / link-rot recovery: Find archived versions of dead inbound links and plan redirects or outreach
• Legal evidence: Retrieve timestamped snapshots of web pages for documentation
• Web archiving: Bulk-check coverage for a list of URLs before deeper archiving work