Pricing

from $8.25 / 1,000 items

Wayback Machine CDX URL List Scraper

Pull every archived URL the Internet Archive has captured for any domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and direct snapshot links. Filter by date range, status, MIME, and uniqueness. Export to JSON, CSV, or Excel for SEO recovery and competitive research.

Pricing

from $8.25 / 1,000 items

Rating

0.0

(0)

Developer

ParseForge

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Categories

Developer tools

Automation

Other

Max Items

maxItems

Optional

Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000.

Type:integer

Minimum:1

Maximum:1000000

URL or domain

url

Optional

Domain or URL to look up in the Wayback Machine. Examples: apify.com, example.com/blog, https://news.ycombinator.com.

Type:string

Match type

matchType

Optional

How to match the URL: exact (exact URL only), prefix (URL plus all subpaths), host (entire host), or domain (host plus all subdomains).

Type:string

Default:domain

Options:

exactprefixhostdomain

From date (YYYYMMDD)

fromDate

Optional

Earliest snapshot timestamp. Examples: 2020, 202001, 20200115. Leave empty for the entire archive.

Type:string

To date (YYYYMMDD)

toDate

Optional

Latest snapshot timestamp. Same format as From date.

Type:string

HTTP status code

statusCode

Optional

Restrict to a specific status code, e.g. 200, 404, 301. Leave empty for all status codes.

Type:string

MIME type

mimeType

Optional

Filter by MIME type, e.g. text/html, application/pdf, image/jpeg. Leave empty for all.

Type:string

Collapse duplicates

collapse

Optional

Collapse adjacent duplicate snapshots. Common values: digest (dedupe by content hash), urlkey (one entry per URL), timestamp:6 (one per month).

Type:string

Unique URLs only

uniqueOnly

Optional

When enabled, returns only the first snapshot per unique URL.

Type:boolean

Default:false

Wayback Machine Archive Scraper

andok/wayback-machine-scraper

Fetch historical snapshots of any webpage from the Internet Archive. Perfect for digital forensics and tracking deleted content.

Andok

Wayback Snapshots — CSV, Date-Filter, Bulk JSON

knotless_cadence/wayback-machine-scraper

Wayback Machine snapshots CSV/JSON — per snapshot: timestamp, status, MIME, size, archive URL — date-filterable. CDX API, no key. 21+ runs. For competitor history-tracking + SEO recovery + brand archaeology. spinov001@gmail.com · blog.spinov.online · t.me/scraping_ai

Alex

Wayback Machine Search

ryanclinton/wayback-machine-search

Search the Internet Archive's Wayback Machine for historical snapshots of any website. Retrieve archived page metadata -- including timestamps, URLs, MIME types, HTTP status codes, and content hashes -- for up to 10,000 snapshots per run.

ryan clinton

Common Crawl URL Index Lookup Scraper

parseforge/common-crawl-index-scraper

Pull every web page Common Crawl captured for a domain or URL prefix. Get timestamps, MIME types, status codes, content digests, and WARC offsets to fetch original payloads. Filter by collection, MIME, and status. Export to JSON, CSV, or Excel for large-scale web research and content discovery.

ParseForge

Federal Court Locator Scraper

parseforge/pacer-court-locator-scraper

Pull every US federal court (district, bankruptcy, appellate, special) with PACER court ID, FJC ID, RSS feed, court URL, jurisdiction code, citation string, and active status. Filter by jurisdiction, state, or court name. Export to JSON, CSV, or Excel for legal directories and court tracking.

ParseForge

Ahrefs Free Tools Scraper

parseforge/ahrefs-tools-scraper

Pull SEO metrics from Ahrefs free tools: Domain Rating, URL Rating, backlinks, referring domains, dofollow backlinks, organic traffic, and organic keywords. Switch between domain and keyword tools. Export to JSON, CSV, or Excel for SEO research, competitor benchmarking, and content audits.

ParseForge

California State Bar Scraper

parseforge/california-state-bar-scraper

Pull California attorney profiles from the public State Bar directory: full name, bar number, license status, address, phone, email, county, district, sections, practice areas, and law school. Search by name or bar number. Export to JSON, CSV, or Excel for legal research and verification.

ParseForge

Bing Search Results Scraper

parseforge/bing-search-scraper

Pull Bing organic search results with rank, title, URL, snippet, displayed URL, deep links, and date. Filter by country, market, language, safe-search, and freshness. Export clean JSON, CSV, or Excel for SEO research, competitor monitoring, content benchmarking, and SERP intelligence.

ParseForge

DuckDuckGo Search Results Scraper

parseforge/duckduckgo-search-scraper

Pull DuckDuckGo organic search results with rank, title, URL, snippet, displayed URL, and favicon. Filter by region, time range, and safe-search. Export clean JSON, CSV, or Excel for privacy-respecting SEO research, competitor discovery, and SERP monitoring with no rate-limit headaches.

ParseForge

Shopify App Store Scraper

parseforge/shopify-app-store-scraper

Pull Shopify apps with name, developer, rating, review count, category, description, screenshots, and icon. Discover via sitemap with 30,000+ apps indexed. Filter by keyword to narrow the results. Export to JSON, CSV, or Excel for ecommerce ecosystem research and competitive analysis.

ParseForge