Internet Archive Scraper & API - archive.org Books, PDFs, Media avatar

Internet Archive Scraper & API - archive.org Books, PDFs, Media

Pricing

from $0.19 / 1,000 results

Go to Apify Store
Internet Archive Scraper & API - archive.org Books, PDFs, Media

Internet Archive Scraper & API - archive.org Books, PDFs, Media

Internet Archive scraper and archive.org search API: keywords, advanced queries, whole collections or item IDs, filtered by media type, year, language. Per item: title, creator, year, subjects, license, downloads, rating, optional PDF/EPUB/MP3 links. $0.19 per 1,000, lowest paid on 16 Nov 2026.

Pricing

from $0.19 / 1,000 results

Rating

0.0

(0)

Developer

Giovanni Rich

Giovanni Rich

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

1

Monthly active users

a day ago

Last modified

Categories

Features

Limited permissions

Share

Internet Archive scraper and unofficial archive.org API: search books, movies, audio, software and images on archive.org and export every item's title, creator, date, subjects, collections, license, download counts, ratings and file formats as JSON, CSV or Excel, with optional direct download links for every PDF, EPUB, MP3 or MP4. It uses archive.org's own public Internet Archive search and metadata APIs over plain HTTP, with no headless browser and no login, so a 100-item search finishes in a few seconds and public domain books and media lists cost very little to build.

What you get

  • Item metadata: identifier, title, media type, creator(s), publisher, description, date, year, language, subjects, collections, ISBN, page count (texts) and runtime (video/audio).
  • Rights: license URL (Creative Commons, public domain mark) and rights statement when the uploader set one.
  • Popularity: all-time downloads, downloads in the last week and month, favorites, review count and average rating.
  • Files (optional): every file in the item with format, source (original/derivative), size, length in seconds, MD5 and a direct https://archive.org/download/... URL. Filter to just pdf, epub, mp3, h.264, ...
  • Links: item page, thumbnail image and download directory for each result.
  • Search control: search in title/subjects/creator (precise, the default) or in archive.org's full text; sort by relevance, downloads, weekly downloads, upload date, publication date, title or rating.

Sample output

Real rows from a test run (queries "apollo 11" and "moby dick").

identifiertitlemediaTypecreatoryeardownloadsavgRating
Apollo11AudioApollo 11audioNASA-2951525
mobydickorwhale01melvuoftMoby-Dick; or, The whaletextsMelville, Herman, 1819-189119221350300
Apollo1116mmOnboardFilmAPOLLO 11 16MM ONBOARD FILMmoviesNASA/Johnson Space Center19693009973.83

Use cases

  • Build reading lists, datasets or catalogs of public-domain books, films and recordings
  • Find openly licensed media (Creative Commons, public domain) for videos, podcasts and AI training sets
  • Research: track what's available on a topic, author or collection, and how popular it is
  • Bulk-download pipelines: get the exact PDF/EPUB/MP3 URLs for a collection and feed them to your downloader
  • Monitor new uploads to a collection on a schedule (sort by Newest uploads)

Pricing

$0.19 per 1,000 items (pay per result; failed requests are never charged), with download links included at no extra charge. Apify platform usage is included in that price (no proxy is needed). Other paid archive.org search actors on the Store charge $0.20 to $21.33 per 1,000, and most add a start fee per run (live Apify Store search, 16 November 2026).

You scrapeYou pay
100 items$0.02
1,000 items$0.19
10,000 items$1.90
100,000 items$19.00

Start fee $0.00005 per run. The Apify free plan's $5 monthly credit covers about 26,000 items. The prefilled input (the 20 most-downloaded "apollo 11" movies) costs about $0.004 and is a single search request (in an October local test, 300 items without file lists took 7.7 s). Turning on file lists adds one request per item but costs nothing extra.

Why this one

  • The lowest paid price of the archive.org item-search actors on the Store we found (live Apify Store search, 16 November 2026): $0.19 per 1,000, download links included, against $0.20 (ludoe11), $0.50 + a start fee (gio21, chorelet), $0.50 (thescrapelab), $0.80 (nonstop_poacher), $1.00 (thriftykiwi, steadydata, jpmarketdata, archive-scout-labs; one_dollar_store, insight.solutions and inovaflow with a start fee), $1.50 (memo23 with a $0.005 start fee, cloud9_ai, haktelaren), $2.00 (dami_studio, quiethand098), $4.00 (thirdwatch), $9.19 (automation-lab), $18 (maged120) and $21.33 (parseforge); about 20 newer ones with 0-1 users charge $0.50-$10.00 (skillmerc, dead00, searchapi, pennydata, firebrick_zamia, starlit_clouds, smilemask, adaptive_nightmare, bakos_bence and others). One free actor exists (environmental_audio). The most-used Internet Archive actors are Wayback Machine tools (ryanclinton 34 users in 30 days, andok 30, logiover 15, caulleonard 13); they return archived web pages, not archive.org items.
  • Full archive.org query syntax: title:, creator:, subject:, collections, years and languages, plus 8 sort orders.
  • Direct download links (optional): every file of an item with format, size and URL, filtered to the formats you want (pdf, epub, mp3).
  • Plain HTTP, no API key: archive.org's own public search API, no proxy cost.
  • Fast default run: the prefilled search is one request to archive.org's search API (300 items took 7.7 s in a local test).

FAQ

Is it legal to scrape the Internet Archive? The actor uses archive.org's public search and metadata APIs, which archive.org provides for this kind of access, and it returns item metadata, not personal data (it even drops users' personal "favorites" lists from the collections field). Many items are public domain or Creative Commons, but others are in copyright, so check each item's license and rights before reusing files. This is not legal advice.

Do I need an archive.org account, login or API key? No. Everything is public and the actor needs no login and no proxy.

How fast is it and how many results can I get? A 100-item search finishes in a few seconds. Up to 10,000 items per query (archive.org's search API limit); split bigger searches by year, media type or collection. File lists add one request per item (300 items with file lists took 308 s in a local test, about 1 second per item). Leave queries empty and set collection to list a whole collection.

Can I schedule it or call it via API? Yes. Schedule a Newest uploads search daily to watch a collection, or start runs from the Apify API, the JavaScript/Python clients, Make, Zapier, n8n or the Apify MCP server.

Will I get blocked? Do I need proxies? Unlikely. archive.org's APIs are open and the actor keeps a polite default concurrency without a proxy. Failed requests (archive.org sometimes returns 5xx under load) are retried with exponential backoff, up to 5 times. If you ever see HTTP 429, lower maxConcurrency or turn on Apify Proxy.

Is there an official Internet Archive API? Yes: archive.org has free public search and metadata APIs (advancedsearch and metadata endpoints). This actor uses them and adds the work around them: paging up to archive.org's 10,000 results per query, one clean row per item, flattened fields, optional per-file download links filtered by format, retries, and CSV / Excel / JSON export, schedules and integrations without writing code.

How much does a typical job cost? A reading list of 200 public-domain books with PDF links is about $0.04. A daily watch on new uploads to one collection (about 50 items a day) is about $0.29 a month. A 10,000-item catalog of a whole collection is $1.90.

Does it scrape Wayback Machine snapshots? No. It searches archive.org items (books, films, audio, software, images, data). For archived copies of web pages, use a Wayback Machine actor.

Example inputs for common jobs

Copy one into the JSON tab of the input and adjust.

Public-domain books by an author with direct PDF and EPUB links (Project Gutenberg collection):

{ "queries": ["creator:\"Jane Austen\""], "collection": "gutenberg", "mediaTypes": ["texts"], "includeFiles": true, "fileFormats": ["pdf", "epub"] }

The 500 most-downloaded films of a whole collection (no query needed; about $0.10):

{ "collection": "prelinger", "sortBy": "downloads", "maxResults": 500 }

Weekly watch on new uploads to a collection (schedule it; deduplicate on identifier):

{ "collection": "nasa", "sortBy": "newestUploads", "maxResults": 100 }

French-language books published 1800-1900:

{ "queries": ["histoire"], "mediaTypes": ["texts"], "language": "fre", "yearFrom": 1800, "yearTo": 1900, "maxResults": 1000 }

Jazz recordings with MP3 links, most downloaded first:

{ "queries": ["subject:jazz"], "mediaTypes": ["audio"], "sortBy": "downloads", "maxResults": 200, "includeFiles": true, "fileFormats": ["mp3"] }

Full metadata and file lists for items you already know:

{ "identifiers": ["Apollo11Audio", "https://archive.org/details/mobydickorwhale01melvuoft"], "includeFiles": true }

How to use

  1. Enter one or more searches in Search queries (e.g. apollo 11, moby dick, or advanced syntax like creator:"Mark Twain").
  2. Optionally pick Media types (texts, movies, audio, software, ...), a Collection (e.g. gutenberg, prelinger, nasa), a year range, a language and a sort order.
  3. Turn on Include file list and download links if you want every file with its format, size, MD5 and download URL.
  4. Click Start and download the results as JSON, CSV, Excel or HTML from the Output tab, or pull them through the Apify API.

You can also skip searching and paste item identifiers or URLs (e.g. https://archive.org/details/Apollo11Audio) to get their full metadata and file lists.

Input example

FieldDefaultDescription
queries-Searches, one per line. Plain words must all match. Advanced syntax works: title:(moby dick), creator:"Mark Twain", subject:jazz
searchInmetadatametadata = title, subjects and creator (precise); title; everything = archive.org full-text search (broad, noisier)
mediaTypesalltexts, movies, audio, software, image, data, web, collection, etree
sortByrelevancedownloads, downloadsThisWeek, newestUploads, oldestUploads, dateNewest, dateOldest, titleAZ, rating
maxResults100Items per query, up to 10,000
collection-Collection ID(s), e.g. gutenberg, prelinger, nasa
yearFrom / yearTo-Publication year range
language-e.g. eng, fre, English
includeFilesfalseAdds the file list with download links (one extra request per item)
fileFormatsallWith file lists on: keep only these formats/extensions, e.g. pdf, epub, mp3
identifiers-Item IDs or archive.org URLs to look up directly

Example:

{
"queries": ["apollo 11", "moby dick", "creator:\"Mark Twain\""],
"mediaTypes": ["movies", "texts", "audio"],
"sortBy": "downloads",
"maxResults": 100,
"includeFiles": true,
"fileFormats": ["pdf", "epub", "mp3", "h.264"]
}

Output example

One dataset item per archive.org item. This one is from a real run (moby dick, sorted by downloads, with fileFormats: ["pdf","epub"]; lists shortened):

{
"query": "moby dick",
"position": 2,
"identifier": "mobydickorwhale01melvuoft",
"title": "Moby-Dick; or, The whale",
"mediaType": "texts",
"creator": "Melville, Herman, 1819-1891",
"date": "1922-01-01",
"year": 1922,
"publisher": "London Constable",
"language": ["eng"],
"collections": ["robarts", "toronto", "university_of_toronto"],
"licenseUrl": null,
"downloads": 135030,
"downloadsLastWeek": 678,
"downloadsLastMonth": 3738,
"favorites": 208,
"itemSizeBytes": 648030033,
"fileFormats": ["DjVuTXT", "EPUB", "MARC", "Text PDF", "hOCR"],
"pageCount": 398,
"publicDate": "2006-12-06T16:57:58Z",
"url": "https://archive.org/details/mobydickorwhale01melvuoft",
"thumbnailUrl": "https://archive.org/services/img/mobydickorwhale01melvuoft",
"downloadUrl": "https://archive.org/download/mobydickorwhale01melvuoft",
"filesCount": 2,
"files": [
{
"name": "mobydickorwhale01melvuoft.epub",
"format": "EPUB",
"source": "derivative",
"sizeBytes": 1433807,
"md5": "c78305e972609a88a73bf61242926249",
"url": "https://archive.org/download/mobydickorwhale01melvuoft/mobydickorwhale01melvuoft.epub"
}
],
"scrapedAt": "2026-09-25T04:55:01.353Z"
}

The dataset has two ready-made views: Overview (thumbnail, title, type, creator, year, downloads, rating, license, link) and Files (formats, file count and file list).

Field fill rates (real test run: 300 items for "apollo 11", "moby dick" and creator:"Mark Twain", movies/texts/audio)

FieldFill rate
identifier, title, media type, collections, downloads, links, dates added100%
subjects97%
description99%
creator89%
date / year78%
language68%
license URL44% (only items whose uploader chose a license)
runtime (video/audio)37%
publisher26%
file list with download URLs100% of items when includeFiles is on (82% had files matching the format filter)

API example

Python (pip install apify-client):

from apify_client import ApifyClient
client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("rel8ble/internet-archive-scraper").call(run_input={"queries": ["creator:\"Mark Twain\""], "mediaTypes": ["texts"], "includeFiles": True, "fileFormats": ["epub"]})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item["identifier"], "|", item["title"], "|", item["year"])

One HTTP call that runs the actor and returns the rows (curl):

curl -X POST "https://api.apify.com/v2/acts/rel8ble~internet-archive-scraper/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
-H "Content-Type: application/json" \
-d '{"queries": ["apollo 11"], "mediaTypes": ["movies"], "sortBy": "downloads", "maxResults": 20}'

Integrations

  • Make, Zapier and n8n: start runs and send new items to your other apps.
  • Google Sheets: export the dataset straight into a spreadsheet, or refresh it on a schedule.
  • Apify API: run the actor and fetch results over REST, or with the official JavaScript and Python clients.
  • Webhooks: get notified when a run finishes and start your download pipeline.
  • Schedules: run a Newest uploads search daily to watch a collection.
  • MCP for AI agents: through the Apify MCP server (https://mcp.apify.com), Claude, ChatGPT, Cursor and other agents can search the Internet Archive with this actor and read the results.

Limits (read before large runs)

  • 10,000 results per query. archive.org's search API won't page deeper. Split big searches by year (yearFrom/yearTo), media type or collection.
  • File lists cost one extra request per item. Items like full audiobooks have hundreds of files, so file-list runs took about 1 second per item in a local test (300 items in 308 s).
  • Metadata quality varies. Everything comes from what uploaders entered: some items have no date, creator or license. The actor returns null rather than guessing.
  • Full-text search is noisy. searchIn: "everything" matches words anywhere, including reviews and OCR text, so popular but unrelated items can rank high when sorting by downloads. The default (title, subjects, creator) avoids this.
  • The actor doesn't download files. It gives you the URLs; download them with your own tool, and respect each item's license.
  • Wayback Machine snapshots (archived websites) are not covered; this actor is for archive.org items.

How it works (for developers)

Searches go to archive.org/advancedsearch.php (the Solr-backed search API) with the fields, sort and paging archive.org documents. Plain-word queries are turned into title:(...) OR subject:(...) OR creator:(...) so results stay on topic; advanced Lucene queries are passed through unchanged. File lists come from archive.org/metadata/{identifier}. Requests run through Crawlee's HttpCrawler with a session pool and exponential-backoff retries, and progress is saved so a migrated run resumes without duplicates.

Run it locally:

npm install
npm test # parser tests
APIFY_LOCAL_STORAGE_DIR=./storage node src/main.js # input in storage/key_value_stores/default/INPUT.json

Need a custom scraper or a ready dataset? Hire me on Fiverr: https://www.fiverr.com/rel8ble