# Internet Archive Metadata Scraper — Bulk archive.org Export (`logiover/internet-archive-metadata-scraper`) Actor Bulk-export item metadata from the Internet Archive (archive.org) by full-text query, collection, media type, creator, subject and date range. Extract identifier, title, creator, date, downloads, format, subject and more. Millions of items. No API key, no login. - **URL**: https://apify.com/logiover/internet-archive-metadata-scraper.md - **Developed by:** [Logiover](https://apify.com/logiover) (community) - **Categories:** Developer tools, Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js/docs.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python/docs.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Internet Archive Metadata Scraper — Bulk archive.org Item Export by Query, Collection & Date Bulk-export **item metadata from the Internet Archive (archive.org)** — texts, audio, video, software, images, live concerts and web collections — filtered by full-text/Lucene **query**, **collection**, **media type**, **creator**, **subject** and **date range**, straight into clean tabular rows. Powered by the **keyless** archive.org **Scrape** and **Advanced Search** APIs, it turns the world's largest public digital library into a spreadsheet or dataset in one run. **No API key, no login, no cookies, no browser.** ![Apify Actor](https://img.shields.io/badge/Apify-Actor-00A67E?logo=apify&logoColor=white) ![No API key](https://img.shields.io/badge/No%20API%20key-required-2ea44f) ![Pay per result](https://img.shields.io/badge/Pricing-Pay%20per%20result-1C7ED6) ![Millions of items](https://img.shields.io/badge/Coverage-Millions%20of%20items-8B5CF6) ![Export](https://img.shields.io/badge/Export-JSON%20%7C%20CSV%20%7C%20Excel-F59E0B) --- ### What does the Internet Archive Metadata Scraper do? This Actor exports **item-level metadata catalogs** from [archive.org](https://archive.org) at scale. You give it a **raw Lucene query** (for example `collection:nasa AND mediatype:audio`) or a set of **structured filters** — collection, media type, creator, subject and a date range — and it paginates the archive's public **Scrape API** by cursor to return up to **millions** of matching items as flat rows. For each item you get `identifier`, `title`, `creator`, `date`, `publicdate`, `year`, `mediatype`, `collection`, `downloads`, `item_size`, `format`, `language`, `subject`, `description` and ready-made `details`/`metadata` URLs. When you request a **sort** (e.g. `downloads desc`), it automatically switches to the **Advanced Search API**. Optionally it enriches each row with a per-item **file list** from the `/metadata/` endpoint. Everything runs over a direct, keyless connection — no account, no OAuth, no client secret. ### Who is it for? - **Data scientists & ML engineers** sourcing large, license-friendly text, audio, image or video corpora for training and evaluation. - **Librarians, archivists & digitization teams** cataloguing holdings, auditing collections and reconciling metadata at scale. - **Academic researchers & digital humanists** assembling reproducible datasets around a topic, creator, era or collection. - **Media researchers, journalists & documentarians** locating public-domain footage, recordings, photos and documents by subject and date. - **Developers & data engineers** who need a keyless, scriptable feed of archive.org item metadata for search indexes, catalogs and pipelines. ### Use cases - **Build a training corpus** — export every `texts` item in a subject or collection with `downloads`, `language` and `format` to shortlist material for an NLP/LLM dataset. - **Catalog a collection** — dump the full item list of a collection (e.g. `prelinger`, `librivoxaudio`, `internetarcade`) into a spreadsheet for inventory and QA. - **Find the most popular items** — sort a query by `downloads desc` to surface the top-viewed films, books or recordings on a topic. - **Time-boxed research slices** — pull all items dated `2010-01-01 TO 2015-12-31` for a creator or subject to study a period. - **Monitor new uploads** — sort by `publicdate desc` and schedule the Actor to track what is newly added to a collection or subject over time. ### Why use the Internet Archive Metadata Scraper? - **Keyless & login-free** — reads archive.org's public Scrape and Advanced Search APIs directly; no API key, OAuth, account or cookies required. - **19 metadata fields per item** — identifier, title, creator, dates, media type, collection, downloads, size, formats, language, subject, description and item URLs, plus optional file lists. - **True bulk export** — cursor pagination over the Scrape API returns up to **millions** of items in a single run (one NASA collection alone holds 200,000+ items). - **Query it your way** — one raw Lucene query, or point-and-click filters (collection, media type, creator, subject, date range) combined automatically with AND. - **Sort when you need it** — request `downloads desc`, `publicdate desc`, `date asc` and the Actor transparently uses the Advanced Search API. - **Clean, export-ready rows** — string-or-array fields are safely normalized to comma-joined values; export to JSON, CSV, Excel, XML or JSONL, or pull via the Apify API and webhooks. Simple **pay-per-result** pricing. ### What data can you extract? Every matching archive.org item becomes one dataset row. Fields marked *(files)* are only populated when **Fetch per-item file list** is enabled. | Field | Type | Description | |-------|------|-------------| | `identifier` | string | Unique archive.org item identifier (used in every URL). | | `title` | string | Item title. | | `creator` | string | Creator or author (comma-joined if multiple). | | `date` | string | Item date (publication / recording date). | | `publicdate` | string | Date the item became public on archive.org. | | `year` | string | Year of the item. | | `mediatype` | string | `texts`, `audio`, `movies`, `image`, `software`, `web`, `data` or `collection`. | | `collection` | string | Collection(s) the item belongs to (comma-joined). | | `downloads` | number | All-time download / view count. | | `itemSize` | number | Total item size in bytes. | | `format` | string | File formats present in the item, e.g. `JPEG, PDF, MP3` (comma-joined). | | `language` | string | Language(s) of the item. | | `subject` | string | Subject / topic tags (comma-joined). | | `description` | string | Item description / abstract. | | `detailsUrl` | string | Public details page: `https://archive.org/details/`. | | `metadataUrl` | string | JSON metadata endpoint: `https://archive.org/metadata/`. | | `fileCount` | number | Number of files in the item. *(files)* | | `topFiles` | string | Sample of up to 10 file names in the item. *(files)* | #### Example row (real archive.org data) ```json { "identifier": "00-042-154", "title": "International Space Station exhibit", "creator": "NASA/Stennis Space Center", "date": "2000-05-26T00:00:00Z", "publicdate": "2009-08-26T16:30:57Z", "year": "2000", "mediatype": "image", "collection": "nasa, stennisspacecentercollection", "downloads": "1376", "itemSize": "1292064", "format": "Archive BitTorrent, Item Tile, JPEG, JPEG Thumb, Metadata", "language": "", "subject": "StenniSphere exhibit, International Space Station, ISS, What -- International Space Station (ISS)", "description": "The International Space Station (ISS) exhibit in StenniSphere at John C. Stennis Space Center gives visitors an up-close look at the largest international peacetime project in history.", "detailsUrl": "https://archive.org/details/00-042-154", "metadataUrl": "https://archive.org/metadata/00-042-154" } ```` ### How to use You can target the archive in two ways. Both accept the same field selection, sort, `maxItems` cap and optional file enrichment. #### Option A — Structured filters (no query language needed) 1. Create a task for this Actor and open the **Input** tab. 2. Fill in any combination of **Collection**, **Media Type**, **Creator**, **Subject** and **Date From / Date To**. They are combined with AND. 3. (Optional) Set a **Sort** like `downloads desc`, choose your **Fields**, and a **Max Items** cap. 4. Click **Start**, then export from the **Output** tab. ```json { "collection": "nasa", "mediaType": "image", "maxItems": 200 } ``` #### Option B — Raw Lucene query (full control) 1. Put a complete archive.org query in the **Query** field — this overrides the structured filters. 2. Combine any fields with `AND` / `OR`, quote phrases, and use ranges like `date:[2015-01-01 TO 2020-12-31]`. 3. Optionally add a **Sort** and **Max Items**, then **Start**. ```json { "query": "grateful dead AND mediatype:audio AND collection:etree", "sort": "downloads desc", "maxItems": 500 } ``` ### Input parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `query` | string | `""` | Raw Lucene / full-text query. If set, used directly and the filters below are ignored. | | `collection` | string | `""` | Collection identifier, e.g. `nasa`, `librivoxaudio`, `prelinger`. Adds `collection:`. | | `mediaType` | select | `any` | One of `any`, `texts`, `audio`, `movies`, `image`, `software`, `web`, `data`, `collection`. | | `creator` | string | `""` | Creator / author filter. Adds `creator:`. | | `subject` | string | `""` | Subject / topic filter. Adds `subject:`. | | `dateFrom` | string | `""` | Start of the item `date` range (`YYYY-MM-DD`). | | `dateTo` | string | `""` | End of the item `date` range (`YYYY-MM-DD`). Applied as `date:[from TO to]`. | | `fields` | array | *(14 default fields)* | Which metadata fields to request. `identifier` is always included. | | `sort` | string | `""` | e.g. `downloads desc`, `publicdate desc`, `date asc`. When set, uses Advanced Search (≤10k). | | `fetchFiles` | boolean | `false` | Also fetch each item's file list from `/metadata/` (one extra request per item; slow). | | `maxItems` | integer | `1000` | Maximum items to export. `0` = unlimited (careful with huge collections). | | `proxyConfiguration` | object | Apify Proxy on | Datacenter proxy by default; a fresh IP is used on retries. | #### Full input example ```json { "query": "", "collection": "librivoxaudio", "mediaType": "audio", "creator": "", "subject": "fiction", "dateFrom": "", "dateTo": "", "fields": ["identifier", "title", "creator", "date", "mediatype", "downloads", "item_size", "format", "language", "subject"], "sort": "downloads desc", "fetchFiles": false, "maxItems": 1000, "proxyConfiguration": { "useApifyProxy": true } } ``` ### Output example A single fully-populated row (with `fetchFiles: true`, so `fileCount` and `topFiles` are present): ```json { "identifier": "cd_greatest-hits_the-grateful-dead", "title": "Skeletons From The Closet: The Best Of Grateful Dead", "creator": "Grateful Dead", "date": "1974-01-01T00:00:00Z", "publicdate": "2021-03-14T09:22:41Z", "year": "1974", "mediatype": "audio", "collection": "etree, GratefulDead, stream_only", "downloads": "48211", "itemSize": "612340288", "format": "VBR MP3, Flac, PNG, Metadata, Archive BitTorrent", "language": "eng", "subject": "Grateful Dead, rock, live, concert", "description": "Compilation of Grateful Dead live and studio recordings.", "detailsUrl": "https://archive.org/details/cd_greatest-hits_the-grateful-dead", "metadataUrl": "https://archive.org/metadata/cd_greatest-hits_the-grateful-dead", "fileCount": "34", "topFiles": "01 - Uncle John's Band.mp3, 02 - Sugar Magnolia.mp3, cover.png, metadata.xml" } ``` ### Tips for best results - **Start narrow, then scale.** Confirm your filters return sensible rows with a small `maxItems` (e.g. 200), then raise the cap for the full bulk export. - **Use `collection:` for the highest signal.** Collections like `nasa`, `librivoxaudio`, `prelinger`, `internetarcade`, `etree`, `opensource_movies` and `gutenberg` are large, well-tagged and reliable. - **Combine `mediaType` with a subject or creator** to slice a big collection into the exact media you want. - **Drop the sort for true bulk.** Sorting uses Advanced Search, which archive.org caps at ~10,000 results. Leave `sort` empty to stream **unlimited** items via the Scrape API. - **Date ranges use the item `date` field**, not the upload date. Use `sort: "publicdate desc"` if you care about newly *added* items instead. - **Request only the fields you need.** Fewer fields = smaller, faster output — great for very large jobs. - **Enable `fetchFiles` only when necessary.** It adds one request per item and is much slower; keep it off for pure metadata catalogs. - **Quote multi-word values** inside a raw query, e.g. `subject:"world war ii"` or `creator:"Mark Twain"`. - **Escape ranges correctly** in raw queries: `date:[2000-01-01 TO 2005-12-31]` (note the spaces and uppercase `TO`). - **Deduping is automatic** — items are de-duplicated by `identifier` within a run. ### Integrations Wire the Actor's dataset into your stack with a few clicks: - **Google Sheets** — push item catalogs straight into a sheet for review and sharing. - **Slack** — get notified when a scheduled run finishes or a monitored collection grows. - **Zapier / Make / n8n** — trigger downstream automations from new archive.org items. - **Webhooks** — POST results to your own endpoint the moment a run succeeds. - **Schedule** — run daily/weekly to keep a fresh mirror of a collection, subject or creator. Export formats include **JSON, CSV, Excel, HTML, XML and JSONL** from the Console or the Apify API. ### API usage Run the Actor programmatically with the Apify API or clients. Replace `` with your Apify API token. **cURL** ```bash curl -X POST "https://api.apify.com/v2/acts/logiover~internet-archive-metadata-scraper/runs?token=" \ -H "Content-Type: application/json" \ -d '{"collection":"nasa","mediaType":"image","maxItems":200}' ``` **Node.js (apify-client)** ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: '' }); const run = await client.actor('logiover/internet-archive-metadata-scraper').call({ query: 'collection:librivoxaudio AND mediatype:audio', sort: 'downloads desc', maxItems: 500, }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items); ``` **Python (apify-client)** ```python from apify_client import ApifyClient client = ApifyClient("") run = client.actor("logiover/internet-archive-metadata-scraper").call(run_input={ "collection": "prelinger", "mediaType": "movies", "maxItems": 1000, }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item["identifier"], item["title"]) ``` ### Use with AI agents (MCP) This Actor works as a tool for AI agents through the **Apify MCP (Model Context Protocol) server**, so an LLM agent can fetch archive.org item metadata on demand. Point your MCP-capable client (Claude, an agent framework, etc.) at Apify's MCP server and expose `logiover/internet-archive-metadata-scraper`. Example prompt: *"Use the Internet Archive Metadata Scraper to export the 300 most-downloaded audio items in the `librivoxaudio` collection, sorted by downloads, and give me title, creator and download count."* ### FAQ #### Do I need an API key, an account or a login to scrape the Internet Archive? No. The Actor reads archive.org's public **Scrape** and **Advanced Search** APIs over a direct connection — no API key, OAuth, account, client secret or cookies are required. #### How much data can I export in one run? The Scrape API paginates by cursor and can return **millions** of items; a single large collection such as `nasa` holds well over 200,000 items. Use `maxItems` to control volume and cost, and leave `sort` empty for unlimited bulk. #### Why is there a ~10,000-result limit sometimes? Only when you request a **sort**. Sorting uses archive.org's Advanced Search API, which is hard-capped at ~10,000 results. Remove the `sort` and the Actor switches to the Scrape API, which has no such cap. #### What media types can I scrape? All Internet Archive media types: `texts`, `audio`, `movies` (video), `image`, `software`, `web`, `data` and `collection`. Filter to one with **Media Type**, or leave it `any`. #### How do I write a query for a specific collection and date range? Either fill in **Collection** + **Date From/To**, or use a raw query like `collection:prelinger AND date:[1950-01-01 TO 1959-12-31]`. Ranges must use uppercase `TO` and `YYYY-MM-DD` dates. #### My run returned zero items — what happened? Usually the filters are too specific or a value is misspelled (collection identifiers are exact, e.g. `librivoxaudio`, not `LibriVox`). Broaden the query, drop a filter, or verify the collection at `archive.org/details/`. #### Why are some fields empty or null? archive.org metadata is contributor-supplied, so not every item has every field (many images lack a `language`, some items lack a `description`). Empty fields are omitted from the row rather than guessed. #### Can I get the list of files inside each item? Yes — enable **Fetch per-item file list**. Each row then includes `fileCount` and `topFiles` from the `/metadata/` endpoint. It is slower (one extra request per item), so keep it off for pure metadata catalogs. #### How do I export the results to CSV or Excel? Open the **Output/Dataset** tab and download as CSV, JSON, Excel, XML, HTML or JSONL, or pull the data through the Apify API. Every item is a flat row, so it imports cleanly into spreadsheets and databases. #### How is this different from the Wayback Machine URL extractor? The Wayback tools return **captured web-page snapshots (URLs)**. This Actor returns the **digital-library item catalog** — the books, audio, film, software and image *items* hosted on archive.org, with their bibliographic metadata. Different data entirely. #### How fresh is the data and how often is it updated? Results are pulled **live** from archive.org on every run, so they reflect the current catalog. Schedule the Actor (e.g. sorted by `publicdate desc`) to keep an up-to-date mirror of a collection or subject. #### Is scraping archive.org allowed? The Actor collects only publicly available metadata via archive.org's own public APIs. You are responsible for using the data in line with the Internet Archive's terms of use and applicable law (see below). ### Is it legal? This Actor extracts only **publicly available metadata** exposed by the Internet Archive's own public Scrape and Advanced Search APIs, and it does not bypass any authentication or access control. It is intended for legitimate research, cataloguing, analytics and dataset-building. You are responsible for complying with the [Internet Archive Terms of Use](https://archive.org/about/terms.php), the individual items' rights and licenses, applicable copyright, and privacy regulations such as GDPR/CCPA when you store or process the results. Scraping public data is generally lawful, but how you use it is your responsibility. ### Related scrapers Building a broader research or dataset pipeline? Pair this with other logiover Actors: - [Wayback Machine URL Extractor](https://apify.com/logiover/wayback-machine-url-extractor) — historical web-page snapshot URLs from the Wayback Machine. - [Reddit Historical Archive Scraper](https://apify.com/logiover/reddit-historical-archive-scraper) — years of old Reddit posts and comments by date via public archives. - [Hacker News Search Scraper](https://apify.com/logiover/hacker-news-search-scraper) — full-text search over Hacker News stories and comments. - 👉 Browse all **[logiover scrapers on Apify Store](https://apify.com/logiover)** — 180+ Actors across data, media, social, research and B2B. # Actor input Schema ## `query` (type: `string`): Raw archive.org Lucene query. If set, it is used directly and the structured filters below are ignored. Examples: `collection:nasa`, `grateful dead AND mediatype:audio`, `subject:"machine learning" AND date:[2015-01-01 TO 2020-12-31]`. Leave empty to build the query from the filters below. ## `collection` (type: `string`): Collection identifier to scrape (adds `collection:`). Popular collections: `nasa`, `librivoxaudio`, `prelinger` (films), `opensource_movies`, `gutenberg`, `internetarcade`, `etree` (live concerts), `metropolitanmuseumofart-gallery`. Browse collections at archive.org/details. ## `mediaType` (type: `string`): Restrict to one Internet Archive media type (adds `mediatype:`). `(any)` applies no media-type filter. ## `creator` (type: `string`): Filter by creator or author (adds `creator:`). Example: `NASA`, `Mark Twain`, `Grateful Dead`. Combined with the other filters using AND. ## `subject` (type: `string`): Filter by subject or topic tag (adds `subject:`). Example: `astronomy`, `jazz`, `world war ii`. Combined with the other filters using AND. ## `dateFrom` (type: `string`): Start of the item `date` range (inclusive). Applied as a Lucene range `date:[from TO to]`. Requires Date To as well. Example: `2010-01-01`. ## `dateTo` (type: `string`): End of the item `date` range (inclusive). Applied together with Date From as `date:[from TO to]`. Example: `2020-12-31`. ## `fields` (type: `array`): Which metadata fields to request per item. `identifier` is always included. Fewer fields = faster + lighter output. ## `sort` (type: `string`): Optional sort, e.g. `downloads desc`, `publicdate desc`, `date asc`, `week desc`. When set, the Actor uses the Advanced Search API (capped at ~10,000 results). When empty, it uses the Scrape API for unlimited bulk export (no sorting). ## `fetchFiles` (type: `boolean`): Also fetch each item's full file list from the /metadata/ endpoint (adds fileCount + top file names + server). This makes ONE extra request per item and is much slower — leave OFF for fast bulk metadata export. ## `maxItems` (type: `integer`): Maximum number of items to export. The Scrape API can return millions of items across cursor pagination, so raise this for large bulk jobs. Set 0 for no cap (careful — some collections hold hundreds of thousands of items). ## `proxyConfiguration` (type: `object`): Proxy configuration. archive.org's public APIs are keyless and work over Apify's datacenter proxy; a fresh proxy IP is used on retries. Residential is not required. ## Actor input object example ```json { "query": "", "collection": "", "mediaType": "any", "creator": "", "subject": "", "dateFrom": "", "dateTo": "", "fields": [ "identifier", "title", "creator", "date", "publicdate", "mediatype", "collection", "downloads", "item_size", "format", "language", "subject", "year", "description" ], "sort": "", "fetchFiles": false, "maxItems": 200, "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `identifier` (type: `string`): Unique archive.org item identifier. ## `title` (type: `string`): Item title. ## `creator` (type: `string`): Creator or author. ## `date` (type: `string`): Item date (publication / recording date). ## `publicdate` (type: `string`): Date the item was made public on archive.org. ## `year` (type: `string`): Year of the item. ## `mediatype` (type: `string`): Media type (texts, audio, movies, image, software, web, data). ## `collection` (type: `string`): Collection(s) the item belongs to (comma-joined). ## `downloads` (type: `string`): All-time download / view count. ## `itemSize` (type: `string`): Total item size in bytes. ## `format` (type: `string`): File formats present in the item (comma-joined). ## `language` (type: `string`): Language(s) of the item. ## `subject` (type: `string`): Subject / topic tags (comma-joined). ## `description` (type: `string`): Item description / abstract. ## `detailsUrl` (type: `string`): Public archive.org details page for the item. ## `metadataUrl` (type: `string`): archive.org JSON metadata endpoint for the item. ## `fileCount` (type: `string`): Number of files in the item (only when Fetch files is enabled). ## `topFiles` (type: `string`): Sample of file names in the item (only when Fetch files is enabled). # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "fields": [ "identifier", "title", "creator", "date", "publicdate", "mediatype", "collection", "downloads", "item_size", "format", "language", "subject", "year", "description" ], "maxItems": 200 }; // Run the Actor and wait for it to finish const run = await client.actor("logiover/internet-archive-metadata-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "fields": [ "identifier", "title", "creator", "date", "publicdate", "mediatype", "collection", "downloads", "item_size", "format", "language", "subject", "year", "description", ], "maxItems": 200, } # Run the Actor and wait for it to finish run = client.actor("logiover/internet-archive-metadata-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "fields": [ "identifier", "title", "creator", "date", "publicdate", "mediatype", "collection", "downloads", "item_size", "format", "language", "subject", "year", "description" ], "maxItems": 200 }' | apify call logiover/internet-archive-metadata-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=logiover/internet-archive-metadata-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Internet Archive Metadata Scraper — Bulk archive.org Export", "description": "Bulk-export item metadata from the Internet Archive (archive.org) by full-text query, collection, media type, creator, subject and date range. Extract identifier, title, creator, date, downloads, format, subject and more. Millions of items. No API key, no login.", "version": "1.0", "x-build-id": "DpIrPTmNsOBAspBOE" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/logiover~internet-archive-metadata-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-logiover-internet-archive-metadata-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/logiover~internet-archive-metadata-scraper/runs": { "post": { "operationId": "runs-sync-logiover-internet-archive-metadata-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/logiover~internet-archive-metadata-scraper/run-sync": { "post": { "operationId": "run-sync-logiover-internet-archive-metadata-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "query": { "title": "Query (raw Lucene / full-text)", "type": "string", "description": "Raw archive.org Lucene query. If set, it is used directly and the structured filters below are ignored. Examples: `collection:nasa`, `grateful dead AND mediatype:audio`, `subject:\"machine learning\" AND date:[2015-01-01 TO 2020-12-31]`. Leave empty to build the query from the filters below.", "default": "" }, "collection": { "title": "Collection", "type": "string", "description": "Collection identifier to scrape (adds `collection:`). Popular collections: `nasa`, `librivoxaudio`, `prelinger` (films), `opensource_movies`, `gutenberg`, `internetarcade`, `etree` (live concerts), `metropolitanmuseumofart-gallery`. Browse collections at archive.org/details.", "default": "" }, "mediaType": { "title": "Media Type", "enum": [ "any", "texts", "audio", "movies", "image", "software", "web", "data", "collection" ], "type": "string", "description": "Restrict to one Internet Archive media type (adds `mediatype:`). `(any)` applies no media-type filter.", "default": "any" }, "creator": { "title": "Creator / Author", "type": "string", "description": "Filter by creator or author (adds `creator:`). Example: `NASA`, `Mark Twain`, `Grateful Dead`. Combined with the other filters using AND.", "default": "" }, "subject": { "title": "Subject / Topic", "type": "string", "description": "Filter by subject or topic tag (adds `subject:`). Example: `astronomy`, `jazz`, `world war ii`. Combined with the other filters using AND.", "default": "" }, "dateFrom": { "title": "Date From (YYYY-MM-DD)", "type": "string", "description": "Start of the item `date` range (inclusive). Applied as a Lucene range `date:[from TO to]`. Requires Date To as well. Example: `2010-01-01`.", "default": "" }, "dateTo": { "title": "Date To (YYYY-MM-DD)", "type": "string", "description": "End of the item `date` range (inclusive). Applied together with Date From as `date:[from TO to]`. Example: `2020-12-31`.", "default": "" }, "fields": { "title": "Metadata Fields", "type": "array", "description": "Which metadata fields to request per item. `identifier` is always included. Fewer fields = faster + lighter output.", "default": [ "identifier", "title", "creator", "date", "publicdate", "mediatype", "collection", "downloads", "item_size", "format", "language", "subject", "year", "description" ], "items": { "type": "string" } }, "sort": { "title": "Sort", "type": "string", "description": "Optional sort, e.g. `downloads desc`, `publicdate desc`, `date asc`, `week desc`. When set, the Actor uses the Advanced Search API (capped at ~10,000 results). When empty, it uses the Scrape API for unlimited bulk export (no sorting).", "default": "" }, "fetchFiles": { "title": "Fetch per-item file list (heavy)", "type": "boolean", "description": "Also fetch each item's full file list from the /metadata/ endpoint (adds fileCount + top file names + server). This makes ONE extra request per item and is much slower — leave OFF for fast bulk metadata export.", "default": false }, "maxItems": { "title": "Max Items", "minimum": 0, "type": "integer", "description": "Maximum number of items to export. The Scrape API can return millions of items across cursor pagination, so raise this for large bulk jobs. Set 0 for no cap (careful — some collections hold hundreds of thousands of items).", "default": 1000 }, "proxyConfiguration": { "title": "Proxy", "type": "object", "description": "Proxy configuration. archive.org's public APIs are keyless and work over Apify's datacenter proxy; a fresh proxy IP is used on retries. Residential is not required.", "default": { "useApifyProxy": true } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```