Wayback Machine — Historical URLs and Snapshot History
Pricing
Pay per event
Wayback Machine — Historical URLs and Snapshot History
Pull every URL the Internet Archive ever captured for a domain, with the dates, HTTP status and content type of each capture. Recover lost pages after a migration, audit a domain's past before buying it, or track when a single page changed.
Pricing
Pay per event
Rating
0.0
(0)
Developer
Benjamin Jerez
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
19 hours ago
Last modified
Categories
Share
Ask the Internet Archive what it ever saw on a domain. Every URL it captured, when, what HTTP status the page returned, what type it was, how large, and whether the content had changed since the previous crawl.
Three questions, three modes: what pages used to exist, how one page changed over time, and what this domain's past looks like before you buy it.
Unique URLs — the sitemap of what existed
The mode you want after a migration or a redesign. One row per distinct page the Archive ever captured, filtered to real pages rather than assets.
Filtering apify.com to text/html, status 200, excluding query strings:
| First seen | Status | URL |
|---|---|---|
| 2007-05-31 | 200 | http://www.apify.com:80/ |
| 2024-05-22 | 200 | https://apify.com/0code0 |
| 2024-05-29 | 200 | https://apify.com/0code0/yellowpages-in-categories/api/client/curl |
That list is where lost pages come from. A site that moved CMS, dropped a language, or retired a blog leaves hundreds of URLs that still hold links and still get requested. The Archive remembers them when the current sitemap does not.
The URL must not contain filter is what makes the output usable. Excluding ?replytocom, /feed/, /wp-json/ and similar removes the crawl noise that otherwise dominates a WordPress site.
Snapshots — how a page changed
Every capture of one URL, or turn on only captures where the content changed and get just the dates it actually changed. That uses the Archive's own content checksum, so twenty identical weekly crawls collapse to one row.
https://apify.com/store, content changes only:
| Captured | Status | Bytes | Content hash |
|---|---|---|---|
| 2019-07-04T11:02:25Z | 200 | 72 893 | TTEN2TCX6TW4… |
| 2019-07-19T10:33:07Z | 200 | 54 607 | QW6RQLSJAIPU… |
| 2019-07-22T22:30:46Z | 200 | 54 696 | YNGGY3JVFB6B… |
The size column reads as a history on its own: that page lost eighteen kilobytes in a fortnight.
Overview — a domain's past in one row
For anyone about to buy a domain, or trying to work out what a competitor used to be.
dereso.fr:
| Field | Value |
|---|---|
firstCapture / lastCapture | 2016-06-23 · 2026-06-16 |
totalCaptures, uniqueUrls | 2 865 · 996 |
yearsCovered | 2016 through 2026, unbroken |
statusBreakdown | 200: 2 009 · 301: 174 · 404: 20 · 405: 5 |
topContentTypes | text/html 743 · image/png 376 · application/javascript 264 |
Eleven unbroken years with a healthy status mix is a domain that was a real site throughout. A gap of several years, a sudden explosion of thin pages, or a status mix dominated by redirects tells the opposite story — and that is exactly what a dropped-domain listing will not tell you.
Who uses this
SEO and migration teams recovering URLs after a replatform, and finding pages worth redirecting rather than losing. Domain investors checking a name's history before bidding. Competitive researchers tracking when a rival changed pricing or messaging. Journalists and legal teams establishing what a page said on a given date. Anyone rebuilding a site whose CMS is gone.
What it is and is not
This reads the Archive's index, not the pages. You get the URL, the date and the metadata of every capture, plus a waybackUrl that opens the archived page itself. It does not download page content.
A capture is not proof the page was live. The Archive records what it received, including 404s and redirects — which is useful, since a page already broken in 2019 is worth knowing about. The status filter is how you separate the two.
warc/revisit is normal. It marks a capture where the content was unchanged from a previous one, so the Archive stored a pointer rather than a copy. Those rows carry no status code of their own, which is why some rows show an empty status.
Coverage is uneven. The Archive crawls popular sites often and obscure ones rarely. An absent URL means it was never captured, not that it never existed.
Source and licence
The Internet Archive's public CDX index at web.archive.org/cdx/search/cdx. No key, no account. Queried live on every run.
One behaviour worth knowing, because it would otherwise look like a bug in this Actor: the CDX index intermittently answers a valid query with an empty response. The same request repeated three times returned results, nothing, then results again. Reporting that as "never archived" would be a serious error, so an empty answer is retried up to five times before it is believed.
Frequently asked
Why does a domain filter return so little?
Because a scheme was included. https://example.com is read as a URL, and matching around a URL is narrower than matching around a domain. Bare domains are passed through as given, and a scheme is stripped automatically.
How far back does it go?
To 1996 in principle. In practice, to the first time anyone crawled that site — 2007 for apify.com, 2016 for dereso.fr.
How many URLs can one run return? As many as you set. The index is paged with a resume key, so a large site is one run rather than many. Captures run into the hundreds of thousands on big domains, which is why the per-input cap exists.
Can I get the page content?
Not from here. Each row carries a waybackUrl pointing at the archived copy, which you can fetch or open.
Why do two rows have the same URL in Snapshots mode? Because the page was captured twice. Turn on only captures where the content changed to keep just the crawls where something was different.
