Wayback Machine URL History & Removed Pages Finder

List every page the Internet Archive's Wayback Machine has captured for a website, with its first and last seen dates, number of months captured, and direct links to the archived copies. Then let the Actor check each URL on the live site and flag pages that have been removed (404, 410, unreachable, or quietly redirected to the homepage).

Just enter a domain. No browser, no proxies, no login — it uses the public Wayback CDX index.

What can it do?

• 🗂️ Full URL inventory from the archive for a whole domain (incl. subdomains), a single host, a path prefix like /blog/, or one exact URL
• 📅 History per URL: firstSeen, lastSeen, captureMonths, last recorded HTTP status, links to the first and last archived copies
• 🪦 Removed pages finder: checks each archived URL on the live site and returns liveStatus (live, redirected, redirected-to-home, not-found, gone, unreachable …) and isRemoved
• 📸 Snapshot mode: every capture of a page, collapsed per day / month / year or only when the content changed
• 🎯 Filters: date range, HTTP status, content type (HTML by default, add PDFs etc.), regex include/exclude
• 🛡️ Built for the archive's quirks: automatic retries with backoff on rate limits (429) and "temporarily offline" outages, paging with resume keys, per-target error report

Use cases

• SEO link reclamation — find old URLs that still have backlinks but now return 404, and redirect them
• Site migrations & redirect maps — get every URL a site ever had, not only what's in today's sitemap
• Content recovery — find and restore lost articles, product pages or PDFs from archived copies
• Expired / aged domain research — see what a domain hosted and when, before you buy it
• Competitor research — see which pages, products or landing pages a competitor launched and retired over time
• OSINT / security — discover forgotten endpoints and files
• AI agents & data pipelines — simple JSON input/output via API or MCP

Input example

{
  "startUrls": ["example.com"],
  "scope": "domain",
  "mode": "urls",
  "maxResultsPerTarget": 100,
  "onlyRemovedPages": true
}

Field	Description
startUrls	Domains or URLs.
scope	domain (incl. subdomains), host, prefix (path), exact.
mode	urls = one row per unique page with history; snapshots = one row per capture.
maxResultsPerTarget	Max rows per entry (default 100).
fromDate / toDate	Capture date range.
statusFilter	200 (default), 2xx3xx, any.
mimeTypes	Default ["text/html"]. Empty = all files.
includeUrlPatterns / excludeUrlPatterns	Regex filters.
checkLiveStatus	Add live status to every row.
onlyRemovedPages	Output only pages that no longer exist.
snapshotGranularity	all, day, month, year, content-change (snapshots mode).
Advanced	maxCdxPages, liveCheckConcurrency, requestTimeoutSecs, maxRetries, retryBudgetSecs (caps time spent waiting on an unavailable archive).

Output example (urls mode with live check)

{
  "target": "apify.com",
  "url": "https://apify.com/",
  "urlkey": "com,apify)/",
  "firstSeen": "2007-05-31T10:15:38Z",
  "lastSeen": "2026-10-01T04:51:28Z",
  "captureMonths": 125,
  "lastStatusCode": "200",
  "mimeType": "text/html",
  "firstArchiveUrl": "https://web.archive.org/web/20070531101538/http://www.apify.com:80/",
  "lastArchiveUrl": "https://web.archive.org/web/20261001045128/https://apify.com/",
  "liveStatusCode": 200,
  "liveFinalUrl": "https://apify.com/",
  "liveStatus": "live",
  "isRemoved": false
}

Snapshot mode rows contain timestamp, capturedAt, archiveUrl, statusCode, mimeType, digest and length. A per-target report is saved in the key-value store record SUMMARY.

Pricing

Pay per event:

• Result row written to the dataset.
• Live check — only when checkLiveStatus or onlyRemovedPages is on, per URL checked on the live site. In "only removed pages" mode, all checked URLs are charged as live checks, but only removed pages are charged as results.

Nothing is charged when the archive has no captures or is unavailable. Set a maximum cost per run and the Actor stops cleanly at the limit.

FAQ

Why do some runs take a while? The Wayback CDX API can be slow for large domains and occasionally rate-limits or goes offline. The Actor retries automatically (up to maxRetries) and reports any problem in SUMMARY.

What does captureMonths mean? The number of distinct months in which the archive captured that URL — a good proxy for how long and how prominently a page existed.

How is "removed" decided? isRemoved is true when the live URL returns 404 or 410, the host is unreachable, or the page now redirects to the site's homepage (a common "soft 404"). Normal redirects to another page are reported as redirected and not counted as removed.

Does it download page content? No — it returns URLs, dates, metadata and archive links. Open lastArchiveUrl to view the archived copy.

Is this allowed? The Actor reads the Internet Archive's public index and makes one lightweight request per URL to the live site when you enable live checks. It collects no personal data. Please use the data responsibly and respect the Internet Archive's terms.

Found a bug or need a feature? Open an issue on the Issues tab.