# Open Library Scraper · Books, Authors, Editions & Subjects (`reapx/open-library-scraper`) Actor

Scrape Open Library books, authors, subjects, editions, and metadata via Open Library API. Fast HTTP scraper charging per returned record with tiered pricing.

- **URL**: https://apify.com/reapx/open-library-scraper.md
- **Developed by:** [Tarek Etman](https://apify.com/reapx) (community)
- **Categories:** Education, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 record returneds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

![reapX — public sources in, addressable records out](https://reapx.dev/reapx.gif)

## Open Library Scraper · Books, Authors, Editions & Subjects

The **Open Library Scraper** is a high-performance HTTP actor designed to extract comprehensive book records, author profiles, publication details, edition metadata, Internet Archive ebook availability, and subjects from the [Open Library API](https://openlibrary.org/developers/api). Whether you are analyzing literary trends, indexing public domain works, enriching bibliographic databases, or tracking author publications, this scraper provides clean, structured camelCase datasets per book work item.

> Maintained by **reapX**. Every row cites the Apify run that produced it — nothing is
> inferred, modelled or filled in, and a field absent from the source is absent from the row.
> The extracted archive for this source is browsable at
> [reapx.dev/data/open-library-scraper/](https://reapx.dev/data/open-library-scraper/) and mirrored as an open dataset on
> [Hugging Face](https://huggingface.co/datasets/reapxdev/open-library-scraper) and
> [Kaggle](https://www.kaggle.com/datasets/reapxdev/open-library-scraper). Questions: reapxdev@proton.me

Maintained by reapx (`reapxdev@proton.me`). Public data only. Every record cites a real observation from openlibrary.org.

***

### Key Features

- **Fast HTTP Execution**: Operating directly over plain HTTP APIs without browser overhead to maximize throughput and minimize cost.
- **Flexible Search Filters**: Search by general keywords, subjects/genres, author names, publication year bands, or language codes.
- **Pay-Per-Event Billing**: Charged strictly per returned record ($0.001/record with tiered volume discounts). Zero charges on empty results or API blocks.
- **Stable Addressable Identifiers**: Emits clean `slug` identifiers (Open Library Work IDs like `OL2784125W`) compatible with deep entity pages.
- **Rich Bibliographic Metadata**: Captures titles, subtitles, primary & contributing authors, ISBNs, edition counts, fulltext scan availability, and cover image IDs.

***

### ⬇️ Input

The scraper accepts JSON inputs to configure search filters and item caps. All parameters are optional, but providing search criteria narrows results to your domain of interest.

| Field Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `query` | String | `"python"` | General search query term (e.g. `python`, `machine learning`, `science fiction`). |
| `subject` | String | `""` | Filter books by subject or genre (e.g. `science_fiction`, `history`, `fantasy`). |
| `author` | String | `""` | Filter books by author name (e.g. `Stephen King`, `Isaac Asimov`, `Mark Lutz`). |
| `firstPublishYear` | String | `""` | Filter by first publication year or range (e.g. `1995` or `[1990 TO 1999]`). |
| `language` | String | `""` | Filter by language code (e.g. `eng`, `fre`, `ger`, `spa`). |
| `maxItems` | Integer | `100` | Maximum number of book records to extract (1 to 1,000). |

#### Example Input Configuration

```json
{
  "subject": "science_fiction",
  "firstPublishYear": "[1990 TO 1999]",
  "language": "eng",
  "maxItems": 100
}
```

***

### ⬆️ Output

Results are saved directly to the actor run's default dataset. Each output record represents one distinct book work object.

#### Dataset Schema Fields

| Field Name | Type | Description | Example |
| :--- | :--- | :--- | :--- |
| `slug` | String | Unique Open Library Work ID without `/works/` prefix. | `"OL2784125W"` |
| `olKey` | String | Full raw Open Library work key path. | `"/works/OL2784125W"` |
| `title` | String | Primary title of the book. | `"Learning Python"` |
| `subtitle` | String | Subtitle of the book if available. | `"Powerful Object-Oriented Programming"` |
| `authorName` | String | Primary author name. | `"Mark Lutz"` |
| `authorNames` | Array | List of all contributing author names. | `["Mark Lutz"]` |
| `authorKeys` | Array | Open Library author keys (OLIDs). | `["OL22643A"]` |
| `firstPublishYear` | Integer | Year the work was first published. | `1999` |
| `editionCount` | Integer | Total number of known editions on Open Library. | `15` |
| `hasFulltext` | Boolean | Whether full text is readable/borrowable on Internet Archive. | `true` |
| `language` | Array | Language codes for available editions. | `["eng"]` |
| `subject` | Array | Tagged subjects and genres. | `["Python (Computer program language)"]` |
| `publisher` | Array | Publishers associated with the work. | `["O'Reilly Media"]` |
| `isbn` | Array | List of ISBN-10 and ISBN-13 strings. | `["0596158106"]` |
| `coverEditionKey` | String | Key of the edition used for the cover. | `"OL24213197M"` |
| `coverI` | Integer | Numeric ID for Open Library Covers API. | `6496660` |
| `ebookAccess` | String | Digital access status (e.g. `borrowable`, `printdisabled`). | `"borrowable"` |
| `ia` | Array | Internet Archive item identifiers. | `["learningpython0000lutz"]` |
| `publicScanB` | Boolean | Whether a public scan is accessible. | `true` |

#### Sample Output Item

```json
{
  "slug": "OL2784125W",
  "olKey": "/works/OL2784125W",
  "title": "Learning Python",
  "subtitle": "Powerful Object-Oriented Programming",
  "authorName": "Mark Lutz",
  "authorNames": [
    "Mark Lutz"
  ],
  "authorKeys": [
    "OL22643A"
  ],
  "firstPublishYear": 1999,
  "editionCount": 15,
  "hasFulltext": true,
  "language": [
    "eng"
  ],
  "subject": [
    "Python (Computer program language)",
    "Computer programming",
    "Object-oriented programming (Computer science)"
  ],
  "publisher": [
    "O'Reilly Media"
  ],
  "isbn": [
    "0596158106",
    "9780596158101"
  ],
  "coverEditionKey": "OL24213197M",
  "coverI": 6496660,
  "ebookAccess": "borrowable",
  "ia": [
    "learningpython0000lutz"
  ],
  "publicScanB": true
}
```

***

### How it works

1. **Parameter Construction**: The actor builds targeted query strings for Open Library's Search API based on your configured `query`, `subject`, `author`, `firstPublishYear`, and `language`.
2. **Paginated Retrieval**: Requests are dispatched sequentially over HTTP with headers configured for optimal performance and rate-limit compliance.
3. **Resilient HTTP Backoff**: Automatic retries with exponential backoff handle transient network errors or HTTP 429 rate limit responses gracefully.
4. **Data Normalization**: Raw API documents are parsed, cleaned, and mapped to addressable `slug` keys and camelCase properties.
5. **Streaming Dataset Output**: Records are pushed to Apify's default dataset progressively as pages complete, ensuring data is saved incrementally.

***

### Code Examples

#### Integration with Python SDK

```python
from apify_client import ApifyClient

## Initialize the client with your Apify API token
client = ApifyClient("YOUR_APIFY_TOKEN")

## Prepare actor input
run_input = {
    "subject": "science_fiction",
    "firstPublishYear": "[1990 TO 1999]",
    "maxItems": 50
}

## Run the actor and wait for completion
run = client.actor("reapx/open-library-scraper").call(run_input=run_input)

## Fetch dataset items
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"Book: {item.get('title')} by {item.get('authorName')} (OLID: {item.get('slug')})")
```

#### Integration with Node.js SDK

```javascript
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
    token: 'YOUR_APIFY_TOKEN',
});

const input = {
    author: 'Stephen King',
    maxItems: 50
};

(async () => {
    const run = await client.actor('reapx/open-library-scraper').call(input);
    const { items } = await client.dataset(run.defaultDatasetId).listItems();
    items.forEach((item) => {
        console.log(`[${item.slug}] ${item.title} - ${item.firstPublishYear}`);
    });
})();
```

***

### ❓ FAQ

#### 1. How is this scraper billed?

This actor uses **Pay-Per-Event (PPE)** pricing. You pay $0.001 per returned book record written to your dataset. Tiered volume discounts automatically apply for higher usage volumes (down to $0.0004/record on Diamond tier). If an API query yields no results or fails, you pay $0.

#### 2. Can I filter by specific subjects or genres?

Yes. Use the `subject` input field with standard Open Library subject slugs such as `science_fiction`, `history`, `fantasy`, `biography`, `computer_science`, or `art`.

#### 3. What is the difference between `slug` and `olKey`?

`slug` is the stripped work identifier (e.g. `OL2784125W`), which serves as a clean, stable addressable entity key. `olKey` is the original raw path returned by Open Library (e.g. `/works/OL2784125W`).

#### 4. Are fulltext scans available for all returned books?

`hasFulltext` indicates whether Internet Archive holds a digitized scan. The `ia` field provides the specific Internet Archive identifier(s) when available.

***

### 💬 Your feedback

We continuously improve our scrapers based on user feedback. If you have feature requests, bug reports, or questions, contact us at **reapxdev@proton.me**.

***

*Disclaimer: Unofficial - not affiliated with Open Library or Internet Archive. Collects public data only. reapx. Contact reapxdev@proton.me.*

### 🧪 Example input

A real, runnable configuration — this is an actual input this Actor has run with.

```json
{
  "subject": "science_fiction",
  "maxItems": 100
}
```

### 📄 Sample output

One real row from a real run of this Actor, unedited.

```json
{
  "slug": "OL2784125W",
  "olKey": "/works/OL2784125W",
  "title": "Learning Python",
  "subtitle": null,
  "authorName": "Mark Lutz",
  "authorNames": [
    "Mark Lutz",
    "David Ascher"
  ],
  "authorKeys": [
    "OL411267A",
    "OL2726848A"
  ],
  "firstPublishYear": 1999,
  "editionCount": 23,
  "hasFulltext": true,
  "language": [
    "ger",
    "eng"
  ],
  "subject": null,
  "publisher": null,
  "isbn": null,
  "coverEditionKey": "OL9497269M",
  "coverI": 1312568,
  "ebookAccess": "borrowable",
  "ia": [
    "einfuhrunginpyth0000lutz",
    "learningpython00lutz",
    "learningpythonth00lutz"
  ],
  "publicScanB": false
}
```

### ⚠️ Run outcomes and error handling

This Actor reports what happened in the run's **status message**, and it always keeps whatever it collected. These are the outcomes you can get and what each one means.

| Outcome | What it means |
|---|---|
| **Success** | Rows were returned and you were charged `record-returned` at $0.001 per row. |
| **No matches** | The source returned nothing for your filters. **Nothing is charged.** Widen the date window or drop a filter. |

#### What is guaranteed either way

- **Every row is pushed as it is built**, not buffered to the end of the run. Anything that buffers output loses everything to a timeout, a block or a migration; this does not.
- **A field absent from the source is absent from the row.** Nothing is inferred, modelled or filled in to make a row look complete.

# Actor input Schema

## `query` (type: `string`):

General search query term to find books on Open Library (e.g., <code>python</code>, <code>machine learning</code>, <code>science fiction</code>). Leaving this empty will use other filter fields or fall back to returning general popular books.<br><br><b>Consequence:</b> Broader search terms return more results; the higher the maxItems setting, the longer the run takes and the more it costs.<br><br>Leave this field empty to skip this filter.

## `subject` (type: `string`):

Filter books by subject or genre (e.g., <code>science\_fiction</code>, <code>fantasy</code>, <code>history</code>, <code>programming</code>).<br><br><b>Empty value:</b> Leave this field empty to refrain from filtering by subject.<br><br><b>Consequence:</b> Setting a specific subject narrows results; higher item limits take longer and cost more.

## `author` (type: `string`):

Filter books by author name (e.g., <code>Stephen King</code>, <code>Isaac Asimov</code>, <code>J.K. Rowling</code>, <code>Mark Lutz</code>).<br><br><b>Empty value:</b> Leave this field empty to include books from all authors.<br><br><b>Consequence:</b> Specifying an author restricts results; higher maxItems increases run duration and costs.

## `firstPublishYear` (type: `string`):

Filter books by first publication year or range (e.g., <code>1995</code> or <code>\[1990 TO 1999]</code>).<br><br><b>Empty value:</b> Leave this field empty to include books published in any year.<br><br><b>Consequence:</b> Restricting year ranges limits total matches; larger limits take longer and cost more.

## `language` (type: `string`):

Filter books by language code (e.g., <code>eng</code> for English, <code>fre</code> for French, <code>ger</code> for German, <code>spa</code> for Spanish).<br><br><b>Empty value:</b> Leave this field empty to include all languages.<br><br><b>Consequence:</b> Language filtering filters out non-matching books; larger limits take longer and cost more.

## `maxItems` (type: `integer`):

Maximum number of book records to scrape and write to the default dataset.<br><br><b>Empty value:</b> Leave empty to use the default limit of 100 items.<br><br><b>Consequence:</b> The higher the number, the longer the run takes and the more it costs.<br><br>The higher the number, the longer the run takes and the more you are charged, because you pay per row returned.

## Actor input object example

```json
{
  "query": "python",
  "maxItems": 100
}
```

# Actor output Schema

## `results` (type: `string`):

Every book record found by this run, one row per item, in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "python"
};

// Run the Actor and wait for it to finish
const run = await client.actor("reapx/open-library-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "query": "python" }

# Run the Actor and wait for it to finish
run = client.actor("reapx/open-library-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "python"
}' |
apify call reapx/open-library-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=reapx/open-library-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/QvWkBY9eDBW5blfXV/builds/GaAVDUrAsoy8W0lFP/openapi.json