Internet Archive Scraper

Search archive.org, read full item metadata and file lists, and list Wayback Machine snapshots for any URL.

What you get

The Internet Archive holds tens of millions of free books, films, concerts, software titles, images, and archived web pages. This Actor turns three of its public endpoints into clean dataset rows.

• Search archive.org with a plain phrase or a fielded query, filtered by media type, collection, creator, subject, language, and year, sorted by relevance, downloads, or date.
• Item metadata for identifiers you already have, including the complete file manifest with formats, sizes, checksums, and direct download URLs.
• Wayback snapshots for a URL, host, or whole domain, with capture timestamps, HTTP status, content digest, and a ready-to-open snapshot link.

Output fields

Field	Description
record_type	item, item_metadata, or wayback_snapshot
identifier, title, creator, description	Core item fields
mediatype, collections, subjects, languages	Classification
date, year, publicdate, addeddate	Item and publication dates
downloads, item_size_bytes	Popularity and size
files, file_count, total_size_bytes	File manifest in item mode
original_url, timestamp, captured_at, status_code, digest	Wayback capture details
snapshot_url, details_url, metadata_url	Direct links back to archive.org

Example output

{"record_type":"item","identifier":"CSPAN3_20190824_120000","title":"20th anniversary special on Apollo 11 moon landing","mediatype":"movies","downloads":58,"collections":["TV-BBC"],"details_url":"https://archive.org/details/CSPAN3_20190824_120000"}

{"record_type":"wayback_snapshot","queried_url":"apify.com","timestamp":"20070531101538","captured_at":"2007-05-31T10:15:38+00:00","status_code":200,"snapshot_url":"https://web.archive.org/web/20070531101538/http://www.apify.com:80/"}

Input parameters

Parameter	Required	Description
mode	Yes	search, item, or wayback
query	No	Phrase or fielded query such as creator:"NASA" AND subject:"mars"
mediaTypes, collection, creator, subject, language	No	Search filters
yearFrom, yearTo	No	Publication-year range
sort	No	Relevance by default; downloads, publish date, size, or identifier
identifiers	For item	archive.org identifiers to describe
maxFilesPerItem	No	Files listed inside each item record; defaults to 50
urls	For wayback	URLs or domains to list captures for
waybackMatchType	No	exact, prefix, host, or domain
waybackCollapse	No	Collapse captures to one per year, month, day, or unique content
waybackFrom, waybackTo, waybackStatusCode	No	Capture date and status filters
maxResults	No	Upper bound on saved rows; defaults to 100

Use cases

• Researchers and librarians can export a subject or collection as a structured catalogue.
• Journalists and legal teams can evidence how a page looked on a given date.
• SEO and brand teams can track when a competitor's pages changed or disappeared.
• ML teams can shortlist public-domain texts, audio, and images with licence fields attached.

Limitations

archive.org caps deep paging at 10,000 results per query, so very broad searches need to be narrowed by year, collection, or media type to be collected fully. Relevance, availability, and metadata quality are the Internet Archive's own; user-contributed items vary. Per-user fav-* pseudo-collections are dropped from collections because a popular item can carry thousands of them. The Actor reads metadata and links only. It does not download the media files themselves, and it does not bypass access restrictions on lending-library items.

Compared to alternatives

Generic web scrapers point at archive.org's HTML and break with every redesign. This Actor uses the documented public search, metadata, and CDX endpoints, so the fields stay stable. It also covers all three surfaces in one place: catalogue search, item manifests, and Wayback captures, which otherwise means three separate tools.

FAQ

Is an API key required?

No. All three endpoints are public and keyless.

How do I search a specific collection?

Set collection to the collection identifier, such as nasa or librivoxaudio, or write it into query as collection:nasa.

Why did my search return fewer rows than the reported match count?

archive.org stops paging at 10,000 results. The run logs the total it saw; narrow the query to reach the rest.

Related Actors

• Google News Scraper - Fast, Pay Per Result, No Proxy — Thirdwatch
• Substack Scraper - Posts, Authors & Search — Thirdwatch
• VK Posts Scraper - Walls, Media & Engagement — Thirdwatch

Last verified: 2026-09

More scrapers at thirdwatch.dev.