Wayback Machine & Internet Archive Scraper – Snapshots, Search

Query the Wayback Machine's own snapshot history, check whether a page is archived at all, and full-text search archive.org's item catalog — all from one Actor, all against archive.org's official, key-less JSON APIs. No login, no browser, no scraping the Wayback Machine's website itself.

Built for SEO and legal teams checking a page's publication/change history for evidence or audits, researchers and journalists tracking how a page or a set of domains evolved over time, and AI agents that need structured archive data mid-task instead of parsing HTML.

Three modes in one Actor, one consistent row shape per mode, and every mode is backed by an API that Internet Archive itself publishes and maintains — nothing here depends on scraping a page that could change its markup tomorrow.

What you get

• Snapshots mode — the full CDX capture history for any URL: every timestamp, status code, MIME type and content digest archive.org has for it, one row per capture.
• Latest mode — the single most recent archived snapshot for a list of URLs, with an available flag so "nothing archived" is a clean false, not an error. A false from archive.org's availability API is double-checked against the CDX index before it is reported (that API can answer "not archived" while it is overloaded); if both endpoints are down you get an unbilled error row, never a silent false negative.
• Search mode — full-text search across archive.org's books, audio, video, software and text items, with title, creator, download count and description per result.
• Errors instead of crashes — a malformed URL or a failed request becomes one unbilled error row (error, errorMessage); a URL with zero snapshots is not an error either, it just produces no row.

Sample rows

Snapshots mode, a real row from a default run:

{
  "original_url": "http://example.com:80/",
  "timestamp": "2002-01-20T14:25:10+00:00",
  "archived_url": "https://web.archive.org/web/20020120142510/http://example.com:80/",
  "status_code": "200",
  "mime_type": "text/html",
  "digest": "HT2DYGA5UKZCPBSFVCV3JOBXGW2G5UUA",
  "length_bytes": 1792,
  "urlkey": "com,example)/",
  "scraped_at": "2026-09-11T09:49:22+00:00"
}

Latest mode, a real row:

{
  "original_url": "example.com",
  "available": false,
  "archived_url": null,
  "timestamp": null,
  "status": null,
  "scraped_at": "2026-09-11T09:52:50+00:00"
}

Search mode, a real row for the query "climate data":

{
  "identifier": "climatechangecom0000unse",
  "title": "Climate change communication and the Internet",
  "media_type": "texts",
  "date": "2017-01-01T00:00:00Z",
  "creator": null,
  "downloads": 182,
  "description": "x, 127 pages : 26 cm The volume provides a timely, state of the art collection of studies examining climate change communication in the era of digital media...",
  "item_url": "https://archive.org/details/climatechangecom0000unse",
  "scraped_at": "2026-09-11T09:52:32+00:00"
}

Filters

Input	Mode	Default	What it does
mode	all	snapshots	snapshots, latest or search.
urls	snapshots, latest	["example.com", "apify.com"]	Domains or URLs to look up. A malformed entry becomes an unbilled error row instead of stopping the run.
from / to	snapshots	unset	Year or date bounds on capture time, e.g. 2024 / 20261231.
matchType	snapshots	exact	exact, prefix, host or domain.
statusCode	snapshots	200	Only include captures with this HTTP status; blank = every status.
maxSnapshotsPerUrl	snapshots	100	Stop after this many capture rows per URL.
collapse	snapshots	digest	Collapse consecutive captures sharing a CDX field value (default: skip re-captures with identical content).
query	search	climate data	Full-text query against archive.org's item catalog.
mediaType	search	unset	Restrict to a media type, e.g. texts, audio, movies, software.
maxResults	search	100	Stop after this many search result rows.

No personal data

Snapshots and latest rows contain only URLs, timestamps and technical metadata that archive.org already publishes. Search rows carry a creator field, which is the catalog's own byline for an uploaded item (an author or publisher name, not personal data collected by this Actor) — otherwise every field is item metadata archive.org makes public.

Pricing

Pay per event, no subscription: $0.50 per 1,000 rows. A default run (2 URLs, snapshots mode, up to 100 captures each) costs a few cents. Error rows are never charged, and a URL with zero snapshots produces no row and no charge.

Use it from your tools

• API and SDKs — call it via the Apify API or the official Python/JavaScript clients: one call to start the run, one to fetch the rows as JSON, CSV or Excel.
• n8n, Make and Zapier — trigger runs and feed the rows into a workflow through Apify's integration for each.
• AI agents and MCP — eligible for agentic use via Apify's MCP server with pay-per-event pricing, so an agent can hand it a URL or a search query mid-task and pay only for the rows it actually got back.

Details

• APIs used: the CDX snapshot index (web.archive.org/cdx/search/cdx), the availability API (archive.org/wayback/available), and the item search API (archive.org/advancedsearch.php) — all official, key-less, documented archive.org endpoints.
• Fetching: plain httpx GET over https only, no proxy, no browser. 429 and 5xx responses are retried with exponential backoff (honouring a Retry-After header when archive.org sends one), up to 5 tries; 404/401/403 are not retried.
• Run stats: the STATS record in the run's key-value store holds per-mode counts, including error rows and (for snapshots mode) URLs with zero captures.
• Not affiliated with the Internet Archive — this Actor is an independent client for its public APIs.