# Public Domain Ebook RAG Feed (`codeclouds/public-domain-ebook-rag-feed`) Actor

Public-domain books for RAG/LLM pipelines from Project Gutenberg's catalog and Open Library. Clean chunked text, author death years and an EU life+70 public-domain status per book.

- **URL**: https://apify.com/codeclouds/public-domain-ebook-rag-feed.md
- **Developed by:** [Dennis](https://apify.com/codeclouds) (community)
- **Categories:** Developer tools, Business
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Public Domain Ebook RAG Feed

Public-domain books for RAG and LLM pipelines, delivered as clean, chunked text with full metadata. Sources: **Project Gutenberg's own catalog feed** (78,000+ texts, clean plain text via its mirror network) and **Open Library / Internet Archive** (public-domain scans with OCR text). Independent tool, not affiliated with these projects. Every book tells you *why* it is considered public domain, and adds something most catalogs don't: an **EU public-domain status** (life + 70 years) computed from the death years of the author *and* translator.

### When should an AI agent use this?

- "Build a RAG corpus of 19th-century English novels, split by chapter."
- "Give me public-domain French or German literature that is also free to use in the EU."
- "Find all Project Gutenberg editions of *Pinocchio* and tell me which translations are still protected in Europe."
- "Fetch clean, boilerplate-free text of *Pride and Prejudice* in ~2,000-character chunks with overlap."
- "List public-domain books on a subject (e.g. 'Science fiction', 'Philosophy') with author birth/death years."

### What this Actor does

- **Project Gutenberg, the official way.** Uses Gutenberg's daily catalog feed (`pg_catalog.csv`) and fetches book text from the official mirror (`gutenberg.pglaf.org`), following Gutenberg's published robot-access policy. It does not crawl the gutenberg.org website and does not depend on third-party wrappers. The Project Gutenberg license header and footer are stripped, so your chunks contain only the book itself.
- **Open Library as second source.** Covers books that are not on Gutenberg. An Open Library record counts as public domain only when Open Library flags it as a full public scan (`public_scan_b` and `ebook_access: public`). Modern in-copyright editions of classic titles are correctly reported as copyrighted. For public-domain scans, the OCR text comes from Internet Archive.
- **No duplicates across sources.** When an Open Library record points to a Gutenberg edition, the cleaner Gutenberg record is used instead.
- **EU public-domain status per book.** `likely`, `not-yet` or `unknown`, based on the death year of every author, editor and translator (illustrators are ignored because only the text is delivered). Example: Collodi died in 1890, but Carol Della Chiesa, who translated the Gutenberg English *Pinocchio*, died in 1972. That translation is public domain in the US and still protected in the EU until 2043. The Actor reports it as `not-yet`.
- **RAG-ready chunking that never drops text.** `chapter` splits on real chapter headings (CHAPTER I, Book Two, …). `paragraph` packs whole paragraphs up to your size limit. `fixed` uses a character window. Every chunk respects `maxChunkChars`, and overlap repeats whole trailing paragraphs instead of cutting mid-word.
- **Fast repeat runs.** The 21 MB Gutenberg catalog is cached for 24 hours in a named key-value store, so scheduled runs don't re-download it.

### Input

| Field | Type | Description |
|---|---|---|
| `search` | string | Words that must all appear in the title or author names (case-insensitive). |
| `languages` | array | ISO 639-1 codes, e.g. `["en", "fr"]`. Empty = all languages. |
| `subjects` | array | Keep books whose subjects or bookshelves contain any of these terms. |
| `copyright` | boolean | `false` (default) = public domain in the US. Full text is only fetched for public-domain books. |
| `euPublicDomainOnly` | boolean | Keep only books that are `likely` public domain in the EU too. |
| `limit` | integer | Max books (default 100, 0 = no limit). |
| `sources` | array | `gutenberg`, `openlibrary` (default: both). `standard-ebooks` is currently skipped. |
| `chunking.enabled` | boolean | `true` (default) = one item per text chunk; `false` = one metadata item per book. |
| `chunking.strategy` | string | `chapter` (default), `paragraph` or `fixed`. |
| `chunking.maxChunkChars` | integer | Maximum characters per chunk (default 2000). |
| `chunking.overlapChars` | integer | Overlap between adjacent chunks (default 200). |
| `mode` | string | `full` (default). `watchlist` is reserved for a future release and currently behaves like `full`. |

### Output

Each dataset item is a text chunk (chunking enabled) or a book (chunking disabled). Every chunk carries the full book metadata, so a RAG pipeline needs no extra lookups.

```json
{
  "bookId": "gutenberg:1342",
  "source": "gutenberg",
  "title": "Pride and Prejudice",
  "author": "Austen, Jane",
  "contributors": [{ "name": "Austen, Jane", "birthYear": 1775, "deathYear": 1817 }],
  "subjects": ["England -- Fiction", "Love stories", "Courtship -- Fiction"],
  "bookshelves": ["Best Books Ever Listings", "Harvard Classics", "Romance"],
  "languages": ["en"],
  "issued": "1998-06-01",
  "copyright": false,
  "copyrightSource": "gutenberg-catalog",
  "publicDomainIn": "US",
  "euPublicDomain": "likely",
  "pageUrl": "https://www.gutenberg.org/ebooks/1342",
  "textUrl": "https://gutenberg.pglaf.org/1/3/4/1342/1342-0.txt",
  "gutenbergIds": ["1342"],
  "chunkIndex": 3,
  "totalChunks": 230,
  "text": "CHAPTER I.\n\nIt is a truth universally acknowledged, that a single man in possession of a good fortune must be in want of a wife. ...",
  "charCount": 3960,
  "chunkStrategy": "chapter"
}
```

Open Library records use `bookId: "openlibrary:<work id>"`, `copyrightSource: "openlibrary-scan-flag"`, an `iaIdentifier`, and `euPublicDomain: "unknown"` (Open Library search results don't include death years).

### Use cases

- Build RAG knowledge bases and retrieval benchmarks from classic literature.
- Assemble multilingual, rights-annotated text corpora for LLM fine-tuning or evaluation.
- Filter a corpus down to works that are safe to use in the EU, not just the US.
- Find every edition and translation of a work, with its contributors and their dates.
- Feed literary or educational AI agents with clean, chapter-aware context.

### Pricing

Pay per event:

- **book-metadata:** $0.002 per book delivered as metadata only (chunking disabled, or no public-domain text available).
- **book-chunk:** $0.0002 per text chunk. A full novel such as *Pride and Prejudice* is about 230 chunks at 4,000 characters (≈ $0.05) or about 400 chunks at the default 2,000 characters (≈ $0.08).

Use `chunking.enabled: false` to browse the catalog cheaply first, then fetch text only for the books you need.

### Legal

- **Public domain is country-specific.** `copyright: false` means public domain *in the United States*. Project Gutenberg and Internet Archive both follow US law. The `euPublicDomain` field applies the EU rule (life + 70 years, counted from 1 January after the year of death) to every author, editor and translator. It is `unknown` whenever a death year is missing or uncertain; the Actor never guesses. It is a good-faith indicator, not legal advice. Check the rules of your own country before commercial use.
- **Sources and terms.** The Project Gutenberg catalog and mirror are used as Gutenberg's robot-access policy describes. Project Gutenberg is credited as the source (`source`, `pageUrl`), which Gutenberg's permission page allows, including for commercial use. "Project Gutenberg" is a registered trademark: if *you* republish texts commercially under that name, Gutenberg's license and royalty terms apply to you. This Actor strips the Gutenberg license header and footer and delivers the underlying public-domain text. This Actor is **not affiliated with or endorsed by Project Gutenberg**, Open Library or Internet Archive. Open Library and Internet Archive public-domain scans are used as published.
- **No personal data.** The output contains bibliographic data about authors and contributors, most of them long deceased.

### FAQ

**Why not Gutendex?**
Gutendex is an unofficial third-party wrapper. It is blocked on many cloud IP ranges, and its text links point at the gutenberg.org website, which Gutenberg asks robots not to crawl. Gutenberg's own catalog feed and mirror give the same books in a more reliable and compliant way.

**Why is a book `unknown` for the EU?**
At least one relevant contributor (author, editor or translator) has no death year in the catalog: an anonymous work, an organisation, or a translator listed without dates. To be safe, those books are not marked as EU public domain.

**How accurate is chapter detection?**
It recognises common headings (CHAPTER/Chapter/BOOK/PART/STAVE followed by a number, Roman numeral or word). Texts without such headings (poetry, essays, plays) automatically fall back to paragraph packing. No text is dropped in either case.

**Does Open Library text look as clean as Gutenberg text?**
No. It is OCR of scanned books, so it can include library stamps, publisher pages and occasional OCR errors. That is one reason Gutenberg editions are preferred when both exist.

**Is an API key needed?**
No. All sources are public and keyless.

### Related Actors

No other live actor from this account pairs directly.

***

*Zoekwoorden: Project Gutenberg, publiek domein, e-books, RAG, tekstcorpus, LLM-training, auteursrecht EU*

### Keywords

public domain, project gutenberg, open library, internet archive, ebook, rag, text corpus, llm training, chunking, eu copyright, life plus 70, literature, translations, metadata, plain text

### Changelog

#### 0.2.0

- Replaced the unofficial Gutendex API (blocked on cloud IPs) with the official Project Gutenberg catalog feed and mirror, used as Gutenberg's robot policy describes; the license header and footer are now stripped.
- Open Library: public-domain status is now based on its public-scan flag instead of a hardcoded value, OCR text comes from Internet Archive, and records are deduplicated against Gutenberg editions.
- New fields: `contributors` (with roles and birth/death years), `euPublicDomain`, `publicDomainIn`, `copyrightSource`, `pageUrl`, `textUrl`, `gutenbergIds`, `iaIdentifier`; `bookId` is now a stable string.
- New input `euPublicDomainOnly`.
- Chunking rewritten: `chapter` now splits on real chapter headings, no text is dropped, every chunk respects `maxChunkChars`, and `fixed` no longer loops when overlap ≥ chunk size.
- Pricing: `book-chunk` is now charged per chunk at $0.0002, which matches the documentation (0.1.0 charged once per book).

#### 0.1.0

- Initial release: Gutendex fetcher with RAG chunking, multi-language/subject filters, and public-domain filtering.

# Actor input Schema

## `search` (type: `string`):

Words that must all appear in the title or author names (case-insensitive). Empty = no text filter.

## `languages` (type: `array`):

Filter by language codes (e.g., \['en', 'fr', 'de']). Empty = all languages.

## `subjects` (type: `array`):

Keep books whose subjects or bookshelves contain any of these terms (e.g., \['Science fiction', 'History']). Empty = all subjects.

## `copyright` (type: `boolean`):

Filter by copyright flag. false = public domain in the US (default), true = not confirmed public domain. Full text is only ever fetched for public-domain books.

## `euPublicDomainOnly` (type: `boolean`):

Keep only books that are likely public domain in the EU too (every author and translator died more than 70 years ago). Books with an unknown death year are excluded. Most useful with the Gutenberg source, which includes death years.

## `limit` (type: `integer`):

Maximum number of books to retrieve. 0 = no limit (fetch all matching).

## `mode` (type: `string`):

full = complete catalog dump (default); watchlist = only new additions since last run (requires persistent storage).

## `sources` (type: `array`):

Which public-domain catalogs to include. gutenberg = the official Project Gutenberg catalog (78,000+ texts, clean plain text via the official mirror, author death years). openlibrary = Open Library search with Internet Archive OCR text for public-domain scans. standard-ebooks is currently skipped (its catalog feed requires a paid membership).

## `chunking` (type: `object`):

Configure text chunking for RAG pipelines. Omit or set enabled=false to output full book text.

## Actor input object example

```json
{
  "copyright": false,
  "euPublicDomainOnly": false,
  "limit": 100,
  "mode": "full",
  "sources": [
    "gutenberg",
    "openlibrary"
  ]
}
```

# Actor output Schema

## `results` (type: `string`):

Results stored in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("codeclouds/public-domain-ebook-rag-feed").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("codeclouds/public-domain-ebook-rag-feed").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call codeclouds/public-domain-ebook-rag-feed --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,codeclouds/public-domain-ebook-rag-feed"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/3WVojnNQ3QO45FLcf/builds/uIKNi9kPVzF2ruSyL/openapi.json