Pricing

from $8.25 / 1,000 items

Wayback Machine CDX URL List Scraper

Pull every archived URL the Internet Archive has captured for any domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and direct snapshot links. Filter by date range, status, MIME, and uniqueness. Export to JSON, CSV, or Excel for SEO recovery and competitive research.

Pricing

from $8.25 / 1,000 items

Rating

0.0

(0)

Developer

ParseForge

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Categories

Developer tools

Automation

Other

You can access the Wayback Machine CDX URL List Scraper programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "maxItems": 10,
12    "url": "apify.com",
13    "matchType": "domain"
14};
15
16// Run the Actor and wait for it to finish
17const run = await client.actor("parseforge/wayback-cdx-scraper").call(input);
18
19// Fetch and print Actor results from the run's dataset (if any)
20console.log('Results from dataset');
21console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
22const { items } = await client.dataset(run.defaultDatasetId).listItems();
23items.forEach((item) => {
24    console.dir(item);
25});
26
27// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Wayback Machine CDX Scraper · Internet Archive URL Lister API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Wayback Machine CDX URL List Scraper API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Wayback Machine CDX URL List Scraper API in Python

Wayback Machine CDX URL List Scraper API through CLI

Wayback Machine CDX URL List Scraper OpenAPI definition

Wayback Machine CDX URL List Scraper API

Wayback Machine Archive Scraper

andok/wayback-machine-scraper

Fetch historical snapshots of any webpage from the Internet Archive. Perfect for digital forensics and tracking deleted content.

Andok

Wayback Snapshots — CSV, Date-Filter, Bulk JSON

knotless_cadence/wayback-machine-scraper

Wayback Machine snapshots CSV/JSON — per snapshot: timestamp, status, MIME, size, archive URL — date-filterable. CDX API, no key. 21+ runs. For competitor history-tracking + SEO recovery + brand archaeology. spinov001@gmail.com · blog.spinov.online · t.me/scraping_ai

Alex

Wayback Machine Search

ryanclinton/wayback-machine-search

Search the Internet Archive's Wayback Machine for historical snapshots of any website. Retrieve archived page metadata -- including timestamps, URLs, MIME types, HTTP status codes, and content hashes -- for up to 10,000 snapshots per run.

ryan clinton

Common Crawl URL Index Lookup Scraper

parseforge/common-crawl-index-scraper

Pull every web page Common Crawl captured for a domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and WARC offsets to fetch original payloads. Filter by collection, MIME, and status. Export to JSON, CSV, or Excel for large-scale web research and content discovery.

ParseForge

Federal Court Locator Scraper

parseforge/pacer-court-locator-scraper

Pull every US federal court (district, bankruptcy, appellate, special) with PACER court ID, FJC ID, RSS feed, court URL, jurisdiction code, citation string, and active status. Filter by jurisdiction, state, or court name. Export to JSON, CSV, or Excel for legal directories and court tracking.

ParseForge

Ahrefs Free Tools Scraper

parseforge/ahrefs-tools-scraper

Pull SEO metrics from Ahrefs free tools: Domain Rating, URL Rating, backlinks, referring domains, dofollow backlinks, organic traffic, and organic keywords. Switch between domain and keyword tools. Export to JSON, CSV, or Excel for SEO research, competitor benchmarking, and content audits.

ParseForge

California State Bar Scraper

parseforge/california-state-bar-scraper

Pull California attorney profiles from the public State Bar directory: full name, bar number, license status, address, phone, email, county, district, sections, practice areas, and law school. Search by name or bar number. Export to JSON, CSV, or Excel for legal research and verification.

ParseForge

Bing Search Results Scraper

parseforge/bing-search-scraper

Pull Bing organic search results with rank, title, URL, snippet, displayed URL, deep links, and date. Filter by country, market, language, safe-search, and freshness. Export clean JSON, CSV, or Excel for SEO research, competitor monitoring, content benchmarking, and SERP intelligence.

ParseForge

DuckDuckGo Search Results Scraper

parseforge/duckduckgo-search-scraper

Pull DuckDuckGo organic search results with rank, title, URL, snippet, displayed URL, and favicon. Filter by region, time range, and safe-search. Export clean JSON, CSV, or Excel for privacy-respecting SEO research, competitor discovery, and SERP monitoring with no rate-limit headaches.

ParseForge

Shopify App Store Scraper

parseforge/shopify-app-store-scraper

Pull Shopify apps with name, developer, rating, review count, category, description, screenshots, and icon. Discover via sitemap with 30,000+ apps indexed. Filter by keyword to narrow the results. Export to JSON, CSV, or Excel for ecommerce ecosystem research and competitive analysis.

ParseForge