Wayback Machine Scraper: Snapshots, Archived URLs & History

Get data from the Internet Archive's Wayback Machine in bulk, in three modes:

1. Snapshots: every capture of a page, with date, HTTP status, file type, size, content hash and the link to the snapshot. Optionally one per day, month or year, or only when the content changed.
2. Archived URLs: every URL of a domain that the archive has ever captured, each once. Recover lost pages, find old URLs for redirect maps, or map a site you are about to buy.
3. History summary: first and last capture, archive age in years, captures per year and years without captures. A quick check of expired domains and of a site's past.

Add the text of archived pages (title, description, language, readable text) to read old versions without opening a browser.

• ✅ No API key: uses the Internet Archive's public CDX API
• ✅ Filters: date range, only HTTP 200, file types (for example text/html or application/pdf), skip CSS/JS/images, URL contains
• ✅ Big domains: pages through results automatically (tested with 12,000 URLs of one domain in 103 seconds)
• ✅ Reliable: waits and retries when the archive is busy or rate-limits
• ✅ Many targets in one run, with a summary per target in OUTPUT
• ✅ Fair billing: per result row; history summaries and page texts are billed separately, only when they were delivered

Input examples

Yearly snapshots of a page, with the text of the first three:

{
    "targets": ["books.toscrape.com"],
    "mode": "snapshots",
    "collapse": "year",
    "includeContent": true,
    "maxContentPerTarget": 3
}

All archived pages and documents of a domain (subdomains included):

{
    "targets": ["toscrape.com"],
    "mode": "urls",
    "excludeAssets": true,
    "maxResultsPerTarget": 10000
}

targets accepts domains, URLs, example.com/blog/* (everything under a path) and *.example.com (all subdomains).

Output examples

Real output from test runs. A snapshot with text (text shortened):

{
    "target": "books.toscrape.com",
    "url": "http://books.toscrape.com:80/",
    "timestamp": "2016-10-28T23:08:07Z",
    "statusCode": 200,
    "mimeType": "text/html",
    "digest": "QSOKEUEELB23XUVZRYFWHGOUHOTW5WOE",
    "lengthBytes": 5269,
    "snapshotUrl": "https://web.archive.org/web/20161028230807/http://books.toscrape.com:80/",
    "rawUrl": "https://web.archive.org/web/20161028230807id_/http://books.toscrape.com:80/",
    "content": {
        "title": "All products | Books to Scrape - Sandbox",
        "description": null,
        "language": "en-us",
        "text": "Books to Scrape We love being scraped!\nHome\nAll products\nBooks\nTravel\nMystery\n…",
        "wordCount": 308,
        "truncated": false
    }
}

An archived URL (mode urls): the first capture of the URL plus a link to its latest capture:

{
    "target": "toscrape.com",
    "url": "http://books.toscrape.com/catalogue/amid-the-chaos_788/index.html",
    "timestamp": "2017-09-25T11:15:22Z",
    "statusCode": 200,
    "mimeType": "text/html",
    "lengthBytes": 4131,
    "snapshotUrl": "https://web.archive.org/web/20170925111522/http://books.toscrape.com/catalogue/amid-the-chaos_788/index.html",
    "latestSnapshotUrl": "https://web.archive.org/web/http://books.toscrape.com/catalogue/amid-the-chaos_788/index.html"
}

A history summary (mode summary):

{
    "target": "books.toscrape.com",
    "archived": true,
    "firstSnapshot": "2016-10-28T23:08:07Z",
    "lastSnapshot": "2026-10-11T08:46:30Z",
    "archiveAgeYears": 10,
    "totalCaptures": 303,
    "capturesPerYear": { "2016": 3, "2017": 19, "2018": 34, "2019": 23, "2020": 23, "2021": 19, "2022": 23, "2023": 23, "2024": 67, "2025": 37, "2026": 32 },
    "yearsWithoutCaptures": [],
    "firstSnapshotUrl": "https://web.archive.org/web/20161028230807/books.toscrape.com/",
    "lastSnapshotUrl": "https://web.archive.org/web/20261011084630/books.toscrape.com/",
    "allSnapshotsUrl": "https://web.archive.org/web/*/books.toscrape.com/"
}

rawUrl is the archived file exactly as it was captured, without the Wayback toolbar.

Pricing

Per result row (snapshot or archived URL), plus a separate price per page text and per history summary. In our tests 12,000 archived URLs took 103 seconds with the default 256 MB memory.

Paid Apify plans pay less: 10% off on Starter, 20% on Scale and 30% on Business and higher plans.

FAQ

Why do I get fewer results than the archive shows? By default only successful captures (HTTP 200) are returned. Turn off "Only successful captures" to also get redirects and error pages. In mode urls each URL appears once.

Why are http:// and https:// or :80 versions both there? The archive stores the URL as it was captured. They are the same page; the snapshot links work for both.

Some sites return nothing. Site owners can ask the Internet Archive to exclude their site; then there is no public data. A target with no captures gives zero rows (or archived: false in the summary).

Is the Wayback Machine sometimes down? Yes, the archive is occasionally offline or slow. The Actor retries busy answers; if the archive stays down, the target gets an error in OUTPUT and you are not charged for it.

More tools from Tidelane Data

• WHOIS Domain Lookup: availability, registrar and expiry dates via RDAP
• Sitemap Extractor: all URLs of a website from its sitemaps, with lastmod
• Website Screenshot API: full-page and mobile screenshots without cookie banners
• Broken Link Checker: find 404s, redirects and dead links

All our tools: apify.com/tidelane

---

Impressum

Levi Griesinger, Tidelane Data
Besigheimer Straße 18
74369, Löchgau, Deutschland
E-Mail: LeviGriesinger@gmail.com