`, and `HTTP_TIMEOUT`.
#### Example Output
```json id="pl7k4v"
{
"identifier": "principleofrelat00eins",
"url": "ARCHIVE_ITEM_DETAILS_URL",
"title": "The principle of relativity; original papers",
"itemTitle": "The principle of relativity; original papers",
"creator": [
"Einstein, Albert, 1879-1955",
"Minkowski, H. (Hermann), 1864-1909"
],
"date": "1920",
"mediatype": "texts",
"collection": [
"mitlibraries",
"greatbooks",
"blc",
"americana"
],
"copyrightStatus": "NOT_IN_COPYRIGHT",
"itemSize": 384734651,
"filesCount": 27,
"files": [
{
"name": "principleofrelat00eins.pdf",
"format": "Text PDF",
"fileFormat": "Text PDF",
"size": 16746808,
"downloadUrl": "ARCHIVE_FILE_DOWNLOAD_URL"
}
]
}
```
This example illustrates the output structure. Actual values depend on the selected item and the available metadata.
Table views use `itemTitle`, `summary`, and `fileFormat` for convenient display, while the raw dataset retains the source field names.
### Input Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
| `mode` | string | `search` | Operation mode: `search`, `search_with_files`, `items`, or `count`. |
| `query` | string | `collection:nasa AND mediatype:movies` | Lucene search query. |
| `mediaTypes` | array | Empty | Optional media-type filters. |
| `collections` | array | Empty | Optional collection filters, with one collection per entry. |
| `fields` | array | Standard metadata fields | Fields requested from search results. |
| `sortBy` | string | Empty | Sort order for supported sorted searches. Blank uses the alternative search method. Sorting is ignored beyond 10,000 results. |
| `identifiers` | array | Empty | Item identifiers or supported item-detail URLs for item mode. |
| `fileFormats` | array | Empty | Optional file-format filters. Matching is case-insensitive. |
| `maxItems` | integer | `500` | Maximum output items per run. Hard cap: 50,000. Requests above 10,000 use the alternative search method. |
| `requestDelayMs` | integer | `500` | Delay between requests. Minimum: 200 milliseconds. |
| `timeoutSecs` | integer | `300` | Maximum run duration. Rows already saved are retained when the run stops. |
| `requestTimeoutSecs` | integer | `30` | Maximum duration for each request. |
### Example Inputs
#### 1. Search NASA Movies
```json id="6p2b1d"
{
"mode": "search",
"query": "collection:nasa AND mediatype:movies",
"sortBy": "downloads desc",
"maxItems": 100,
"requestDelayMs": 500,
"timeoutSecs": 300,
"requestTimeoutSecs": 30
}
```
#### 2. Retrieve Item Metadata and PDF Links
```json id="7ks6dt"
{
"mode": "items",
"identifiers": [
"principleofrelat00eins"
],
"fileFormats": ["Text PDF"],
"maxItems": 20
}
```
#### 3. Count Matching Items
```json id="4y9jqs"
{
"mode": "count",
"query": "mediatype:audio AND subject:jazz"
}
```
### Output Fields
| Field | Type | Description |
|---|---|---|
| `identifier` | string | Unique item identifier. |
| `url` | string | Item details-page reference. |
| `title` | string | Original item title. |
| `itemTitle` | string | Title duplicated for convenient table display. |
| `creator` | array | Creator names. A single creator string is normalized into a one-item list. |
| `date` | string | Original date value. |
| `year` | integer | Year parsed from the date when possible. |
| `mediatype` | string | Media category, such as texts, movies, audio, or software. |
| `collection` | array | Associated collection identifiers. |
| `downloads` | integer | Download count, when available. |
| `description` | string | Description text. |
| `summary` | string | Description duplicated for convenient display. |
| `subject` | array or string | Subject metadata, when available. |
| `language` | array or string | Language metadata, when available. |
| `licenseUrl` | string | License reference, when provided. |
| `copyrightStatus` | string | Copyright-status metadata from the item record. |
| `publicDate` | string | Public date, when available. |
| `addedDate` | string | Date the item was added, when available. |
| `uploader` | string | Uploader information, when available. |
| `publisher` | string | Publisher information, when available. |
| `contributor` | string or array | Contributor information, when available. |
| `itemSize` | integer | Total item size, when available. |
| `filesCount` | integer | Number of files associated with the item. |
| `server` | string | Server metadata, when available. |
| `thumbnailUrl` | string | Constructed thumbnail reference. |
| `files` | array | File metadata, including names, formats, sizes, checksums, and download links. |
| `total` | integer | Result count in count mode. |
| `error` | string | Error code for failed rows. |
| `message` | string | Additional error details, when available. |
Fields depend on the selected mode and the metadata available for each item.
### How It Works
1. **Builds the query:** Combines the provided Lucene expression with optional media-type and collection filters.
2. **Selects the search method:** Uses sorted, paginated search for supported requests with sorting and limits of up to 10,000 items. Other searches use cursor-based retrieval.
3. **Retrieves results:** Processes result pages or cursors until the requested item limit, end of results, or timeout.
4. **Enriches item records:** The `search_with_files` and `items` modes retrieve additional metadata and file information.
5. **Builds file links:** Constructs download links using item identifiers and URL-encoded filenames. It does not download the files.
6. **Deduplicates identifiers:** Avoids writing duplicate item records.
7. **Handles failures:** Records errors and continues where possible.
8. **Saves output:** Writes item metadata, file details, counts, and errors to the dataset.
### Limits and Important Notes
- **Sorted search limit:** Sorted, paginated searches stop at 10,000 results. Deeper pagination can return a `DEEP_PAGING` error.
- **Alternative search method:** Larger or unsorted searches use cursor-based retrieval.
- **Minimum request size:** The alternative search method requires a minimum requested count of 100. For smaller `maxItems` values, the actor may retrieve at least 100 records and retain only the requested number.
- **Run limit:** The maximum configured output is 50,000 items.
- **Request pacing:** The default delay is 500 milliseconds, with retries and backoff for HTTP 429 and 5xx errors. Individual waits can reach 60 seconds.
- **Moving collections:** Items added or removed while a cursor is being processed may be skipped.
- **Thumbnail references:** Thumbnail URLs are constructed from item identifiers rather than fetched as separate requests.
- **File downloads:** Only download links are provided. The underlying files are not downloaded.
- **Licensing:** Licenses and copyright status vary by item. Review the metadata and applicable terms before reusing files.
### FAQ
#### Do I need an Internet Archive API key?
No. The actor does not require an API key or login.
#### Does this actor download files?
No. It records file metadata and download URLs. You can use those links in a separate workflow if you are authorized to download and reuse the files.
#### Why did a sorted search stop at 10,000 results?
Sorted, paginated searches have a 10,000-result limit. For larger searches, increase `maxItems` beyond 10,000 or leave `sortBy` empty to use the alternative retrieval method.
#### Why does a small search still request 100 rows?
The alternative search method has a minimum request count of 100. The actor can trim the returned results to match your configured `maxItems`.
#### Can I use the results for AI training?
That depends on the individual item's license, copyright status, and applicable restrictions. A status such as `NOT_IN_COPYRIGHT` is useful metadata, but it should not be treated as a blanket guarantee that every file or associated element is free to reuse. Check the relevant rights before including content in a training dataset.
#### Why are some metadata fields missing?
Not every item has complete metadata. Fields such as creator, publication date, language, license, or file details may be missing when they were not provided in the underlying record.
### Trademark Notice
Internet Archive and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the data source and does not imply affiliation, sponsorship, endorsement, or partnership.
### Disclaimer
This actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or otherwise officially associated with Internet Archive.
### Is Web Scraping Legal?
Collecting information that is already public is generally legal. Scraping is not the same thing as hacking, and there is no single law that bans it. The limits are the usual ones: do not break into accounts, do not take private or confidential data, and be careful with personal data and copyrighted works. A site's terms can also restrict how you access it. Facts such as a public name, price, or rating are not the same thing as republishing someone's article or photo.
This actor collects information from public pages. It does not log in and it does not bypass access controls.
# Actor input Schema
## `mode` (type: `string`):
search lists matching items, search_with_files adds each item's files and download links, items reads identifiers or archive.org URLs, and count returns one total.
## `query` (type: `string`):
Lucene query, for example title:(climate) AND mediatype:texts AND year:\[1900 TO 1950]. Combined with media types and collections.
## `mediaTypes` (type: `array`):
Keep items whose mediatype is one of these. Added to the query as mediatype:(a OR b).
## `collections` (type: `array`):
Keep items in these archive.org collections. Added to the query as collection:(x OR y). One collection per line.
## `fields` (type: `array`):
Advanced Search and Scrape API fields. identifier is always requested. One field name per line.
## `sortBy` (type: `string`):
Sorting is limited to the first 10,000 results. Leave this empty to use the Scrape API. Deep scraping ignores page-based sorting.
## `identifiers` (type: `array`):
Item identifiers or URLs such as https://archive.org/details/principleofrelat00eins. Used when mode is items. One per line.
## `fileFormats` (type: `array`):
When file lists are loaded, keep only files whose format matches, for example Text PDF, MPEG4, or VBR MP3. Matching is case-insensitive. One format per line.
## `maxItems` (type: `integer`):
Free cap 50,000. More than 10,000 uses the Scrape API automatically.
## `requestDelayMs` (type: `integer`):
Pause between archive.org requests. Minimum 200 ms. Default 500 ms.
## `timeoutSecs` (type: `integer`):
Stop the run after this many seconds and keep rows already saved.
## `requestTimeoutSecs` (type: `integer`):
How long to wait for one archive.org response before treating it as a timeout.
## `proxyConfiguration` (type: `object`):
Optional proxy for archive.org requests. Leave Apify Proxy off unless your IP is blocked.
## Actor input object example
```json
{
"mode": "search",
"query": "title:(climate) AND mediatype:texts AND year:[1900 TO 1950]",
"collections": [
"nasa"
],
"fields": [
"identifier",
"title",
"creator",
"date",
"mediatype",
"collection",
"downloads",
"description",
"subject",
"language",
"licenseurl"
],
"sortBy": "",
"identifiers": [
"principleofrelat00eins"
],
"fileFormats": [
"Text PDF"
],
"maxItems": 500,
"requestDelayMs": 500,
"timeoutSecs": 300,
"requestTimeoutSecs": 30,
"proxyConfiguration": {
"useApifyProxy": false
}
}
```
# Actor output Schema
## `items` (type: `string`):
Matching archive.org items with identifier, title, creator, date, media type, downloads, and details URL.
## `files` (type: `string`):
Items loaded with file lists, size, copyright status, thumbnail URL, and details URL.
## `errors` (type: `string`):
Queries or identifiers that failed, with the error code and message.
## `dataset` (type: `string`):
Every row from the run, including count rows, without a table view applied.
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"query": "collection:nasa AND mediatype:movies"
};
// Run the Actor and wait for it to finish
const run = await client.actor("zerobreak/internet-archive-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = { "query": "collection:nasa AND mediatype:movies" }
# Run the Actor and wait for it to finish
run = client.actor("zerobreak/internet-archive-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"query": "collection:nasa AND mediatype:movies"
}' |
apify call zerobreak/internet-archive-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"type": "http",
"url": "https://mcp.apify.com/?tools=fetch-actor-details,zerobreak/internet-archive-scraper"
}
}
}
```
The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).
## OpenAPI specification
Download the OpenAPI definition: https://api.apify.com/v2/actors/OdGjXEkX5taYiOjbD/builds/PPIeBtkhPSeVUz5TC/openapi.json