Wayback Machine Scraper: Snapshots, Page Text & Domain History

Everything the Internet Archive's Wayback Machine holds about a URL or domain, as JSON, CSV or Excel: every capture with date, status and size, the archived page itself as clean text, Markdown or raw HTML for any date, a domain history report (first and last seen, captures per year, homepage titles over time, redirects, parked periods) and archive.org item search; no login, no API key, no browser.

Try it: Click Start with the default input: 100 snapshots of apify.com (one per month, 2007 to 2024) in 10 s for about $0.05 (run on October 21, 2026).

Five of those 100 rows (each row also has the 14-digit timestamp, content digest and a raw id_ link without the Wayback toolbar):

Captured	URL as archived	Status	MIME	Bytes	Archive link
2007-05-31 10:15	http://www.apify.com:80/	200	text/html	1,013	web.archive.org/web/20070531101538/http://www.apify.com:80/
2014-05-17 01:14	http://apify.com/	200	text/html	580	web.archive.org/web/20140517011437/http://apify.com/
2020-06-02 12:52	https://apify.com/	200	text/html	32,641	web.archive.org/web/20200602125235/https://apify.com/
2023-10-02 05:29	https://www.apify.com/	301	text/html	427	web.archive.org/web/20231002052924/https://www.apify.com/
2024-07-03 00:58	https://apify.com/	200	text/html	48,388	web.archive.org/web/20240703005823/https://apify.com/

Pricing: $0.50 per 1,000 snapshots; archived page content $1.50 per 1,000 pages; domain history reports $6 per 1,000 domains; archive.org items $0.50 per 1,000; plus $0.00005 per run. Platform usage is included; failed runs, error rows and URLs with no capture are not charged.

It uses only the Internet Archive's free public endpoints and paces itself the way archive.org asks, so large jobs are deliberately not fast (about one request per second per host).

Real example: what was parse.com before and after Parse?

Domain history mode, input {"mode": "domainHistory", "urls": ["parse.com"], "maxTitleSamples": 20}, cloud run of October 13, 2026 (UTC). Excerpt of the one dataset row:

{
  "domain": "parse.com",
  "firstSeen": "1996-11-03T04:51:13Z",
  "lastSeen": "2026-10-10T18:49:03Z",
  "ageYears": 29.9,
  "homepageCaptures": 9814,
  "yearsWithCaptures": 30,
  "missingYears": [2010],
  "longestGapMonths": 25,
  "redirectTargets": ["parseplatform.org"],
  "parkedYears": ["2008"],
  "lastLiveTitle": "Parse",
  "lastLiveCapture": "2017-01-15T04:51:07Z",
  "summary": "Archived 1996-11-03 to 2026-10-10; 9814 homepage captures in 30 years; no captures in 1 year(s) (2010); 9 distinct homepage title(s); homepage redirected to parseplatform.org in 6 sampled year(s) (2018-2026); parked, for-sale or placeholder page in 2008."
}

The distinctTitles field of the same row tells the story in one glance:

Years seen	Homepage title
1996	PARSE Software Devices
1998–1999	PARSE Software Devices, QNX and Real-Time Consulting
2001–2002	PARSE Software Devices -- Training, Consulting and Books for the QNX and Neutrino Marketplace
2004–2005	PARSE Software Devices -- QNX Books, QNX Training, QNX Consulting, QNX Software, and more!
2007	QNX Books, QNX Training, QNX Consulting, ... by PARSE Software Devices!
2008	Domain Name for Sale (flagged in parkedYears)
2011–2017	Parse
2018–2020	Parse + Open Source = ❤️ (served by parseplatform.org after a redirect)
2021–2024	Parse Platform

A QNX consultancy for 12 years, a for-sale page, the Facebook-owned Parse backend, then a redirect to the open-source project. That is exactly what you want to know before you buy an expired domain or judge a backlink source. The row also has titleHistory (one sampled homepage per year with status, final URL, redirect target, language, word count and archive link) and capturesPerYear.

What it does

Mode	You give	You get	Typical use
Snapshot list (snapshots)	URLs, folders, hosts or domains	One row per capture: time, URL, HTTP status, MIME type, content digest, size, replay link and raw link	Find every archived URL of a site, recover deleted pages, build redirect maps, check when a page changed
Archived page content (pages)	URLs plus dates (2018-06-01, 2019, first, latest, every-year) or Wayback links	The capture closest to each date: title, meta description, H1s, language, canonical, clean text, Markdown, raw HTML, links	Feed old pages to an LLM, restore lost content, compare a competitor's homepage year by year
Domain history (domainHistory)	Domains	First and last seen, age, captures per year, gaps, homepage title per year, redirects to other sites, parked or for-sale signals, optional count of archived URLs	Expired-domain vetting, SEO audits, M&A and brand due diligence, spotting spammy past owners
archive.org search (archiveSearch)	A query, media type, collection	Item metadata (title, creator, date, downloads, collections, subjects) and optionally every file with a direct download URL	Research datasets, public-domain books and films, software archives

More real output (October 13, 2026 runs)

Snapshot list, input {"urls": ["apify.com"], "collapse": "month", "maxSnapshotsPerUrl": 100}: 100 rows in 21 seconds, the first one:

{
  "query": "apify.com",
  "originalUrl": "http://www.apify.com:80/",
  "timestamp": "20070531101538",
  "capturedAt": "2007-05-31T10:15:38Z",
  "statusCode": 200,
  "mimeType": "text/html",
  "digest": "EE6FCHP3MKBC3EV5D5Q4WQJNZNVUTNU6",
  "lengthBytes": 1013,
  "archiveUrl": "https://web.archive.org/web/20070531101538/http://www.apify.com:80/",
  "rawUrl": "https://web.archive.org/web/20070531101538id_/http://www.apify.com:80/"
}

Archived page content, input {"urls": ["apify.com"], "dates": ["2018-06-01", "first"]} (shortened):

{
  "url": "apify.com",
  "requestedDate": "2018-06-01T00:00:00Z",
  "snapshotTimestamp": "20180531124559",
  "daysFromRequested": -0.5,
  "originalUrl": "https://www.apify.com/",
  "statusCode": 200,
  "title": "Apify - The web scraping and automation platform",
  "metaDescription": "Apify extracts data from websites, crawls lists of URLs and automates workflows on the web. Turn any website into an API in a few minutes!",
  "h1": ["The web scraping and automation platform"],
  "internalLinkCount": 28,
  "externalLinkCount": 16,
  "wordCount": 649,
  "text": "The web scraping\nand automation platform\nThe easiest way to extract structured data from any website run headless Chrome / Puppeteer jobs in the cloud ...",
  "parkedSignal": null
}

The first row of the same run found apify.com's oldest capture, from May 31, 2007: an IIS "Under Construction" page, flagged with "parkedSignal": "Under Construction".

Pricing

Pay per result. Platform usage is included, so this is the whole bill.

Event	Price	Per 1,000
Snapshot (one CDX capture row)	$0.0005	$0.50
Archived page (text / Markdown / HTML of one capture)	$0.0015	$1.50
Domain history report (one domain, up to 40 sampled homepages)	$0.006	$6.00
archive.org item (with file list if enabled)	$0.0005	$0.50
Actor start	$0.00005	per run

Not charged: URLs with no capture, never-archived domains, error rows and duplicate captures (two requested dates that resolve to the same snapshot). Set a maximum charge per run in Apify and the actor stops cleanly when it is reached.

Input examples

Every archived URL of a site (one row per distinct URL, HTML pages that answered 200):

{ "mode": "snapshots", "urls": ["example.com"], "matchType": "domain", "collapse": "urlkey",
  "statusCodes": ["200"], "mimeTypes": ["text/html"], "maxSnapshotsPerUrl": 50000 }

Only the captures where a page's content changed, in 2023 and 2024:

{ "mode": "snapshots", "urls": ["https://example.com/pricing"], "collapse": "digest", "dateFrom": "2023", "dateTo": "2024" }

The 20 most recent captures: {"mode": "snapshots", "urls": ["example.com"], "newestFirst": true, "maxSnapshotsPerUrl": 20}

A competitor's homepage once a year as Markdown:

{ "mode": "pages", "urls": ["competitor.com"], "dates": ["every-year"], "dateFrom": "2015", "contentFormats": ["markdown"] }

Vet a list of expired domains fast (capture statistics only, about a second each):

{ "mode": "domainHistory", "urls": ["domain1.com", "domain2.net"], "maxTitleSamples": 0 }

Public-domain NASA films, most downloaded first, with download links:

{ "mode": "archiveSearch", "searchQuery": "apollo 11", "mediaType": "movies", "collection": "nasa",
  "searchSort": "downloads", "maxItems": 50, "includeFileList": true }

Filters and options

• matchType: exact, prefix (a folder), host, or domain (domain plus all subdomains).
• collapse: one capture per hour, day, month or year; one per distinct URL (urlkey); or only when content changed (digest, which compares each capture with the previous one).
• statusCodes: 200, 404, 301, or classes such as 3xx. mimeTypes: text/html, application/pdf, image/*.
• extraFilters: raw CDX filter expressions for power users, for example !original:.*\.(css|js)$.
• dates (pages mode): YYYY, YYYY-MM, YYYY-MM-DD, 14-digit timestamps, first, latest, every-year. You get the capture closest to the moment you ask for, and daysFromRequested says how far away it was.
• contentFormats: text (main content, menus and footers removed), markdown, html (the original bytes as captured, no Wayback toolbar). includeLinks adds every link with anchor text, internal/external and rel (for example nofollow).
• maxTitleSamples (domain history): how many archived homepages to open, one per year spread over the domain's life. Each takes 2 to 4 seconds; 0 gives capture statistics only.
• uniqueUrlsUpTo (domain history): also count distinct archived URLs up to a cap.

How it works, accuracy and limits

• Sources: the Wayback CDX server (web.archive.org/cdx/search/cdx) for snapshot lists, id_ raw snapshot URLs for page content, the Wayback calendar's capture summary for domain statistics, and archive.org/advancedsearch.php plus /metadata for item search.
• Politeness: by default at most one request per second per Internet Archive host, with automatic back-off on 429 and 5xx answers (honouring Retry-After). If archive.org keeps answering 429 the actor slows itself further. Runs are slower than a naive scraper on purpose; it keeps your runs, and everyone else's, from getting blocked.
• The CDX server is slow on big queries. A query can take 10 to 30 seconds. Domain-wide queries on huge sites (google.com, digg.com) can time out at the archive's side; the actor stops retrying after a few timeouts and returns an error row that suggests a narrower query (date range, collapse, or a folder prefix) instead of burning your time.
• Domain history counts are homepage captures, not every URL on the domain. Homepage titles are sampled (one capture per year, the busiest month), so a title that lived for a few months between samples can be missed. Raise maxTitleSamples for more detail.
• Parked and for-sale detection is phrase based ("domain for sale", "buy this domain", parking services, default server pages; weaker phrases like "coming soon" only count on near-empty pages). Treat it as a signal to look, not a verdict.
• Redirects: a homepage that redirected to another registrable domain is reported in redirectedTo / redirectTargets. Redirects to its own subdomains are not.
• Text extraction removes navigation and boilerplate. On link-heavy pages (old portals, search homepages) the main text can be mostly menu items; use html if you need everything.
• Sites excluded from the Wayback Machine (by owner request) return no captures. The archive only has what it crawled; very new or obscure pages may have no captures at all.
• This actor reads the archive; it does not submit pages to Save Page Now.

FAQ

Do I need an Internet Archive account or API key? No. All endpoints used are public.

How is this different from the Wayback Machine website? The website shows one URL at a time. Here you can list tens of thousands of captures with filters, pull the content of hundreds of old pages as clean text in one run, and vet a whole list of domains, all exported as JSON, CSV or Excel or sent to your own systems through the Apify API, webhooks and integrations.

Can I get every page of a website that no longer exists? Run snapshots with matchType: domain, collapse: urlkey, statusCodes: ["200"] to get the URL list, then feed those URLs to the pages mode.

Can AI agents use it? Yes. It works through the Apify API and the Apify MCP server; the Markdown output is ready for LLM context.

Why did a requested date return a capture from a different day? You get the capture closest in time to what you asked for. Check daysFromRequested; if it matters, list the captures first with the snapshots mode and request exact timestamps.

My run hit HTTP 429 or 503 retries. The Internet Archive is shared infrastructure and is sometimes overloaded. The actor backs off and retries on its own. For very large jobs, keep requestIntervalSecs at 1 or higher and avoid starting many runs at the same moment.

Changelog

• 0.1.7 (October 21, 2026): README leads with a dated snapshot run and the four prices; input prefill unchanged. No scraping changes.
• 0.1 (October 13, 2026): First release. Four modes: snapshot list (CDX with match types, date range, status, MIME and raw filters, collapse, newest first, resume-key paging), archived page content (text, Markdown, raw HTML, meta, links, every-year), domain history reports (statistics, sampled homepage titles, redirects, parked signals, URL count) and archive.org item search with file lists.