Internet Archive Scraper

What does Internet Archive Scraper do?

Two tools in one Actor, both powered by archive.org's public APIs:

1. Search mode — query the Internet Archive's collections (books, audio, video, software, web) with full Lucene syntax and export item metadata at scale: identifier, title, media type, dates, download counts, creator, subjects.
2. Wayback mode — list the historical snapshots of any URL from the Wayback Machine: when each capture happened, HTTP status, MIME type and a direct link to the archived page.

Why scrape the Internet Archive?

• Dataset building — bulk metadata for research corpora (books, radio, concerts, old software).
• Web history & compliance — prove what a page said and when it changed.
• SEO & domain research — a domain's full capture history in one CSV.
• Digital humanities — the Archive is the largest public library ever assembled; this gives you its card catalog, structured.

How do I use it?

1. Click Try for free — the prefilled search works as-is.
2. Pick a mode: a search query (collection:librivoxaudio, subject:brasil AND mediatype:texts) or a URL for snapshot history.
3. Run and download JSON/CSV — or connect the Apify API / MCP server.

How much will it cost?

$1.00 per 1,000 records ($0.001 each) plus a $0.005 start fee. Typical research pulls (1,000–20,000 records) cost $1–20; a full 100,000-record corpus costs $100.

⬇️ Input

{
  "mode": "search",
  "query": "subject:brasil AND mediatype:texts",
  "maxItems": 1000
}

{
  "mode": "wayback",
  "url": "apify.com",
  "fromDate": "20200101",
  "maxItems": 500
}

⬆️ Output (real examples)

{
  "record_type": "item",
  "identifier": "brasilcolonial00unse",
  "url": "https://archive.org/details/brasilcolonial00unse",
  "title": "Brasil colonial",
  "mediatype": "texts",
  "downloads": 1243,
  "creator": "…",
  "scraped_at": "2026-09-18T01:00:00+00:00"
}

{
  "record_type": "snapshot",
  "identifier": "20200101000000/https://apify.com/",
  "url": "https://web.archive.org/web/20200101000000/https://apify.com/",
  "original_url": "https://apify.com/",
  "snapshot_at": "2020-01-01T00:00:00+00:00",
  "status_code": "200",
  "mimetype": "text/html",
  "scraped_at": "2026-09-18T01:00:00+00:00"
}

Limitations & fair use

• Search mode returns metadata, not the files themselves — the identifier gives you the item page and download URLs follow from it.
• Wayback mode collapses identical captures (same digest) to keep results meaningful; very large domains can have long CDX response times.
• The Internet Archive is a nonprofit library: this Actor uses their documented public APIs politely. Please don't hammer.

FAQ

Is it legal?

Yes — both APIs are public and documented by the Internet Archive for programmatic use. All data returned is already public.

Can I get the actual page content of a snapshot?

This Actor returns the snapshot index (fast, cheap). Fetching and parsing full archived pages is a heavier job — tell us in the Issues tab if you need it and we'll consider a companion Actor.

Something broke

Open an issue in the Issues tab — canary-tested daily, fixes ship fast.

Your feedback

If this Actor saves you time, a review helps other users find it.