Pricing

from $0.20 / 1,000 archived urls

Try for free

Go to Apify Store

Wayback Machine URL Extractor – Archived URLs, Snapshots & Diff

Try for free

List every URL the Internet Archive Wayback Machine has saved for a domain or URL prefix: first and last capture, capture count, status, MIME type and snapshot links. Date, status and MIME filters. Optional archived page text as Markdown and a content diff between two dates.

Pricing

from $0.20 / 1,000 archived urls

Rating

0.0

(0)

Developer

Cemal Atakli

Actor stats

Bookmarked

Total users

Monthly active users

7 days ago

Last modified

Categories

SEO tools

Developer tools

Domains or URLs

urls

Required

Any mix of: a domain (example.com: every archived URL of the domain and its subdomains), *.example.com, a URL prefix (example.com/blog/ or example.com/docs/*: everything under that path) or an exact page URL (https://example.com/pricing: that page only, useful for the diff). Change how entries are read with Match type.

Type:array

Default:

[
  "crawlee.dev"
]

Match type

matchType

Optional

auto: bare domain → domain incl. subdomains, URL ending in / or * → prefix, other URLs → exact page. Or force one type for all entries: domain (domain + all subdomains), host (this host only, no subdomains), prefix (all URLs starting with the entry) or exact.

Type:string

Default:auto

Options:

autodomainhostprefixexact

Max URLs (total)

maxUrls

Optional

Stop after saving this many unique archived URLs in total (0 = no limit). You pay only for URLs saved. Large domains can have millions of archived URLs.

Type:integer

Minimum:0

Default:200

Max URLs per domain/URL

maxUrlsPerInput

Optional

Limit per entry when you give several (0 = no per-entry limit).

Type:integer

Minimum:0

Default:0

Captured from

from

Optional

Only captures on or after this date. 2015, 2015-06, 2015-06-30 or relative (1 year, 6 months, 30 days).

Type:string

Captured until

Optional

Only captures on or before this date (inclusive: 2019 means until 2019-12-31). Same formats as above.

Type:string

HTTP status codes

statusCodes

Optional

Keep only captures with these original HTTP statuses, e.g. 200, or 3xx for all redirects. Empty = all statuses (incl. 404s and redirects that the crawler recorded).

Type:array

Default:

[]

MIME types

mimeTypes

Optional

Keep only captures with these content types, e.g. text/html, application/pdf, image/*. Empty = all.

Type:array

Default:

[]

Only URLs matching (regex)

urlRegex

Optional

Java regex that must match the whole original URL, e.g. .*\.pdf or .*/blog/.*. Applied by the Internet Archive server, so filtered URLs are never downloaded or charged.

Type:string

Exclude URLs matching (regex)

excludeUrlRegex

Optional

Drop URLs whose whole original URL matches, e.g. .*\.(css|js|png|jpg|gif|svg|woff2?).

Type:string

Skip URLs with query strings

skipQueryStrings

Optional

Drop URLs containing ? (tracking parameters, search pages, session IDs). Useful for clean page lists.

Type:boolean

Default:false

Skip tracking-parameter URLs

skipTrackingUrls

Optional

Drop URL variants whose query contains analytics or session parameters (utm_*, __hstc, fbclid, gclid, sessionid, PHPSESSID…). Real dynamic pages such as ?id=12 or ?p=345 are kept. Filtered on the Internet Archive server, never charged.

Type:boolean

Default:true

Skip malformed URLs

skipMalformedUrls

Optional

Drop obviously broken URLs the archive crawler picked up from bad HTML (containing ", <, >, %22, braces or whitespace). Not charged.

Type:boolean

Default:true

Capture statistics (first/last capture, count, versions)

captureStats

Optional

On: reads every capture to give firstCapture, lastCapture, captureCount, uniqueVersions (distinct content digests), the latest status and a direct link to the latest snapshot. Off: one CDX row per URL (collapse by urlkey), faster for huge domains, but only the first capture is known.

Type:boolean

Default:true

Max captures counted per URL

maxCapturesPerUrl

Optional

Home pages can have 100,000+ captures. After this many, counting stops for that URL (captureCountCapped: true), the newest capture is looked up directly and the scan moves on. Keeps runs fast.

Type:integer

Minimum:10

Default:5000

Fetch snapshot content

fetchContent

Optional

Download each URL's archived page (raw id_ mode, without the Wayback toolbar) and add its title and text. Charged per snapshot downloaded (snapshot-fetched). Non-text captures (images, PDFs) are skipped and not charged.

Type:boolean

Default:false

Content format

contentFormat

Optional

markdown: main content as clean Markdown with links (good for LLMs). text: plain text. html: the raw archived HTML.

Type:string

Default:markdown

Options:

markdowntexthtml

Which snapshot

contentSnapshot

Optional

Content of the newest successful (HTTP 200) capture, or the earliest one.

Type:string

Default:latest

Options:

latestearliest

Max content characters

maxContentChars

Optional

Content longer than this is cut (contentTruncated: true).

Type:integer

Minimum:1000

Default:100000

Compare: older date

compareFrom

Optional

Set both compare dates to get a content diff per URL: the capture closest to each date is downloaded, converted to Markdown and compared (similarity, lines added/removed, unified diff). Up to 2 snapshot-fetched events per URL. Use exact page URLs or a small Max URLs. Formats: 2020, 2020-06-30, 2 years.

Type:string

Compare: newer date

compareTo

Optional

The newer date for the diff, e.g. 2025-01-01 or 1 day.

Type:string

Max diff characters

maxDiffChars

Optional

The unified diff is cut after this many characters (diffTruncated: true). addedLines / removedLines keep up to 200 lines each.

Type:integer

Minimum:500

Default:20000

Delay between CDX requests (seconds)

cdxDelaySecs

Optional

Minimum pause between Internet Archive CDX API calls. The Actor already slows down automatically on HTTP 429/503 and retries with backoff; raise this if you run many jobs in parallel.

Type:integer

Minimum:1

Maximum:30

Default:1

Snapshot request timeout (seconds)

requestTimeoutSecs

Optional

Timeout for downloading one archived snapshot. Failed requests are retried with backoff.

Type:integer

Minimum:10

Maximum:300

Default:60

Wayback Machine URL Extractor - Archived URLs

logiover/wayback-machine-url-extractor

Extract every archived URL of any domain from the Internet Archive's Wayback Machine (CDX API). Recover lost or old pages, build redirect maps and run OSINT, with date and status filters. No API key, export to CSV or JSON.

Logiover

Wayback Machine Snapshots — Archive History

alaudinburki/wayback-machine-snapshots

List all Wayback Machine (archive.org) snapshots for any URL — timestamp, status, MIME, and direct snapshot links — via the public CDX API. Date-range and duplicate-collapse options. No auth.

alaudin burki

Wayback Machine Scraper

glassventures/wayback-machine-scraper

Scrape Wayback Machine archive snapshots for any URL or domain. Get archived URLs, timestamps, status codes, MIME types. Export to JSON, CSV, Excel.

Glass Ventures

Wayback Machine Scraper — Archived Snapshots

hipersoft/wayback-machine-scraper

List every Internet Archive (Wayback Machine) snapshot of a URL or whole domain: timestamp, snapshot URL, status code, MIME type and content digest. Filter by date, status and dedupe. For SEO, OSINT and historical research. No setup.

hiper soft

Wayback Machine Scraper API - Archived URLs & Website History

pink_comic/wayback-machine-archived-urls-website-history

Search the Internet Archive Wayback Machine CDX index for archived URLs and website history. Filter captures by exact URL, prefix, host/domain, date, HTTP status, MIME type, and duplicate rule. Get timestamps, replay URLs, content hashes, status codes, sizes, and source evidence.

Ava Torres

Wayback Machine Checker — Snapshot Diff Alerts

taroyamada/wayback-machine-checker

Compare Wayback snapshots and detect changes. No API key / no login. Open Input → leave defaults → Start (1 URL ≈ USD 0.008): https://console.apify.com/actors/fy1UnkQNIzGyy8HIs/input From USD 8/1k.

naoki anzai

Wayback Machine Archive Scraper

andok/wayback-machine-scraper

Fetch historical snapshots of any webpage from the Internet Archive. Perfect for digital forensics and tracking deleted content.

Andok

167

Sitemap Extractor – All URLs from sitemap.xml + Status Check

gazidev/sitemap-url-extractor

Extract every URL from any website's sitemap.xml: auto-discovery via robots.txt, nested sitemap indexes, .xml.gz, image/news/video/hreflang tags and lastmod. Filter by date or glob, get only new URLs, and optionally check HTTP status, redirects and canonical per URL.

Cemal Atakli

Wayback Machine Snapshot Lookup

nuhemugames/wayback-snapshot-lookup

Full archive history of any URL from the Internet Archive's official CDX API: every snapshot with date, archive link, HTTP status and content digest - or a per-URL summary with first/last capture, snapshots per year and unique content versions. Polite rate control built in. No credentials needed.

kuon

Wayback Machine & Internet Archive Scraper – Snapshots, Search

rowfeed/wayback-internet-archive-scraper

Query the Wayback Machine's official JSON APIs: snapshot history (CDX) for any URL, the latest archived snapshot, and full-text search of archive.org items. No login, no scraping, no proxy.