Internet Archive Collection Intelligence

Turn public Archive.org items into structured catalog, collection, file, engagement, and review data. The Actor accepts known URLs or identifiers as well as ISBN, creator, collection, subject, and advanced-search discovery inputs.

The original flat review dataset remains the default, so existing integrations keep working. New output modes support catalog research, collection analysis, downloadable-file inventories, and recurring change detection.

No Internet Archive API key is required for public data.

Common use cases

• Enrich library or publishing catalogs with identifiers, creators, subjects, collections, and rights metadata.
• Rank or monitor public items by total, weekly, or monthly downloads when Archive.org provides those fields.
• Export public file inventories without downloading the files.
• Build datasets of public reviews and star ratings.
• Detect new or edited reviews, metadata changes, and download growth between successful runs.
• Feed normalized Archive.org records into databases, dashboards, and AI-agent workflows.

Quick start: flat reviews

{
  "sources": ["https://archive.org/details/goodytwoshoes00newyiala"],
  "maxItems": 1,
  "maxReviewsPerItem": 10
}

This uses reviews_flat, the backward-compatible default. Each dataset row is one review enriched with item metadata.

Item and collection intelligence

Use items when you want one row per Archive.org item, including items without reviews:

{
  "collections": ["internetarchivebooks"],
  "outputMode": "items",
  "onlyItemsWithReviews": false,
  "maxItems": 25,
  "maxReviewsPerItem": 10
}

Use items_with_nested_reviews to keep one item row with its fetched reviews attached. Use normalized to emit separate item, review, and optional file rows that are easier to load into related database tables.

{
  "creators": ["Mark Twain"],
  "outputMode": "normalized",
  "onlyItemsWithReviews": false,
  "includeFiles": true,
  "maxItems": 10
}

includeFiles exports public file metadata and download URLs. It does not download file contents.

Discovery inputs

• sources: Archive.org item URLs, metadata URLs, review URLs, identifiers, or ISBNs.
• isbns: ISBN-10 or ISBN-13 values.
• creators: author or creator names.
• collections: Archive.org collection identifiers.
• subjects: subject terms.
• searchQueries: raw Archive.org advanced-search expressions.

Searches can match items without public reviews. Set onlyItemsWithReviews to false for catalog, collection, file, or engagement research.

Output modes

Mode	Dataset shape	Best for
reviews_flat	One enriched row per public review	Existing integrations, sentiment and review research
items	One row per item	Catalog and collection analysis
items_with_nested_reviews	One item row with a reviews array	Document databases and item snapshots
normalized	Separate item, review, and optional file rows	Warehouses, relational pipelines, AI tools

Item output can include identifiers, creators, descriptions, publishers, dates, subjects, collections, media types, rights, licenses, file counts, sizes, download activity, favorites, and review aggregates. Availability depends on the source item.

Change monitoring

Monitoring is opt-in. Reuse a private monitoringKey to compare a source set with its last fully successful snapshot:

{
  "sources": ["goodytwoshoes00newyiala"],
  "outputMode": "normalized",
  "onlyItemsWithReviews": false,
  "monitoringMode": "compare_and_update",
  "monitoringKey": "public-domain-books-weekly"
}

The first successful run creates a baseline. Later successful runs add change_summary rows and label emitted item/review records as new, changed, or unchanged.

The Actor stores only compact hashes, public identifiers, timestamps, download counts, and review identity hashes in its Actor-owned named key-value store. It does not store full review text in monitoring state. A partial or failed run reports state_not_updated and does not replace the last good baseline.

Keep these rules in mind:

• Use the same Apify account and monitoringKey for a continuing comparison series.
• Use a separate key for a different customer, project, source set, or filter configuration.
• Do not infer deletions from a capped or filtered run; change summaries describe the records that were fetched.
• Monitoring does not create a schedule. Run the Actor again manually, by API, or through your own Apify Task when needed.

Review filters and reliability controls

Advanced inputs include minStars, maxStars, reviewTextContains, languageFilter, mediatypes, concurrency, retry, delay, and timeout controls. Raw source objects can be included for debugging, but normalized output is recommended for stable integrations.

The Actor uses public Archive.org HTTP endpoints, applies bounded retries and polite request delays, deduplicates items and reviews, and writes RUN-STATS to the run key-value store.

Example output

Item row:

{
  "entityType": "item",
  "source": "internet_archive",
  "identifier": "goodytwoshoes00newyiala",
  "title": "Goody Two-Shoes",
  "creators": ["Newbery, John"],
  "collections": ["americana"],
  "downloads": 12500,
  "reviewCount": 5,
  "itemUrl": "https://archive.org/details/goodytwoshoes00newyiala"
}

Review row:

{
  "entityType": "review",
  "identifier": "goodytwoshoes00newyiala",
  "reviewTitle": "Fun",
  "reviewText": "This is an enjoyable read.",
  "stars": 4,
  "reviewerName": "ErniePye",
  "itemUrl": "https://archive.org/details/goodytwoshoes00newyiala"
}

API example

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("thescrapelab/internet-archive-book-reviews-scraper").call(run_input={
    "collections": ["internetarchivebooks"],
    "outputMode": "items",
    "onlyItemsWithReviews": False,
    "maxItems": 25,
})

for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(row["identifier"], row.get("downloads"))

Limits and responsible use

• The Actor reads public Archive.org metadata and reviews only.
• It does not log in, alter Archive.org data, download book files, solve captchas, or bypass access controls.
• Some records omit ratings, reviewer names, timestamps, engagement counts, rights, or file properties.
• Endpoint latency and availability can vary.
• Optional Internet Archive credentials remain accepted for compatibility but are not used for public extraction.

Troubleshooting

• No rows in reviews_flat: the item may have no public reviews or a filter removed them. Try outputMode: "items" and onlyItemsWithReviews: false.
• No file rows: use outputMode: "normalized" with includeFiles: true.
• First monitoring run shows no changes: it is establishing the baseline; rerun with the same key to compare.
• Unsupported URL: use an Archive.org /details/{identifier}, /metadata/{identifier}, or /metadata/{identifier}/reviews URL.
• Slow broad search: lower maxItems, use a more specific query, and keep concurrency modest.

Cost guidance

The Store shows the current price before each run. Cost primarily grows with items searched and fetched, review pages requested, file metadata volume, and retry activity. Start with a direct item, maxItems: 1, and a small review limit; expand only after confirming the output fits your use case.