# Wikimedia Commons Book Text Scraper (`parseforge/wikimedia-commons-book-text-scraper`) Actor

Scrapes OCR text from DjVu and PDF books on Wikimedia Commons. Returns each page's title and full text. Supports category and search queries.

- **URL**: https://apify.com/parseforge/wikimedia-commons-book-text-scraper.md
- **Developed by:** [ParseForge](https://apify.com/parseforge) (community)
- **Categories:** Education, Developer tools, Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.62 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

[![ParseForge](https://raw.githubusercontent.com/ParseForge/apify-assets/main/banner.jpg)](https://apify.com/parseforge?fpr=vmoqkp)

### Wikimedia Commons Book Text Scraper

**Scrape book text from Wikimedia Commons DjVu and PDF files.** Every page returns its title and file metadata. No login or API key. Export to CSV, JSON, Excel, or XML.

Wikimedia Commons hosts millions of scanned books as DjVu and PDF files, but extracting the text page by page is tedious. This Actor uses the Commons API to list files by category or search, then fetches the OCR text for each page, returning a clean dataset of page titles and their full text content.

| Who uses it | What they scrape Wikimedia Commons for |
|---|---|
| Digital humanities researchers | Extract text from public domain books for corpus analysis or transcription projects. |
| Data scientists | Build training datasets from scanned book text for NLP models. |
| Librarians and archivists | Bulk download OCR text from Commons book collections for preservation or indexing. |
| Language learners and educators | Gather text from historical or foreign-language books for study materials. |

### What it does

This Actor collects book text from Wikimedia Commons DjVu and PDF files, returning each page as a row with its title and extracted text.

- 📄 **Page-level text extraction:** Each page of a DjVu or PDF file is returned as a separate row with its full OCR text.
- 📂 **Category or search-driven:** Scrape all files in a Commons category (e.g., 'DjVu\_files') or use a custom search query to target specific books.
- 🔍 **File type filter:** Choose between DjVu or PDF files when no category is specified.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

### What you can do with Wikimedia Commons data

**📚 Build a corpus of public domain books.**

A researcher runs the Actor on the 'DjVu\_files' category to download OCR text from thousands of scanned books for linguistic analysis.

**🔍 Search for specific book content.**

A data scientist uses a search query like 'index: filetype:djvu Shakespeare' to extract text only from Shakespeare-related books.

**📊 Archive book text for offline access.**

A librarian scrapes a Commons category of PDF files to create a local text archive of historical documents.

**🤖 Train an OCR correction model.**

An NLP engineer collects page text from Commons DjVu files to build a dataset for training a model that improves OCR accuracy.

### Why choose this scraper

| | What you get |
|---|---|
| **Page text** | Full OCR text for each page of the book. |
| **Page title** | The title of the page on Commons (e.g., 'Page:Example.djvu/1'). |
| **File metadata** | File name, category, and search context. |

### What a Wikimedia Commons record looks like

Every record returns as one flat JSON row. Here is a real one from a run:

```json
{
 "pageid": 40316238,
 "title": "File:Deledda - Cartolina autografa.djvu",
 "fileUrl": "https://commons.wikimedia.org/wiki/File%3ADeledda%20-%20Cartolina%20autografa.djvu",
 "scrapedAt": "2026-09-04T01:20:45.594Z"
}
```

Every value above comes from a real run. A field a record does not have comes back as `null`.

### Configure the run

Drive the Actor from a Wikimedia Commons category name, a search query, or a file type selection. Filters run as each file is listed, so only matching pages reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

```json
{
 "maxItems": 10,
 "category": "DjVu_files",
 "fileType": "djvu"
}
```

A larger pull:

```json
{
 "maxItems": 200,
 "category": "DjVu_files",
 "fileType": "djvu"
}
```

### Free users

Free-plan runs return up to 10 results as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect more results per run.

### Run it

1. [Create a free Apify account](https://console.apify.com/sign-up?fpr=vmoqkp).
2. Open the [Wikimedia Commons Book Text Scraper](https://apify.com/parseforge/wikimedia-commons-book-text-scraper?fpr=vmoqkp).
3. Set your inputs and any filters, then click **Start**.
4. Export the results as CSV, Excel, JSON, or XML from the **Dataset** tab.

Run it programmatically through the [Apify API](https://docs.apify.com/api/v2) (`run-sync-get-dataset-items`) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python.

### Use with AI agents (MCP)

Give an AI agent live access to Wikimedia Commons through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

```bash
claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/wikimedia-commons-book-text-scraper"
```

Then prompt it in plain language to run the scraper and read back the results.

### Troubleshooting

**Why am I getting no results?**

Check that the category name or search query is correct. Category names are case-sensitive (e.g., 'DjVu\_files' not 'djvu\_files'). Also ensure the category contains DjVu or PDF files.

**The Actor returns empty text for some pages.**

Some files on Commons may not have an embedded OCR layer. The Actor will return an empty string for those pages. Try a different file or category.

**The run is taking too long.**

Large categories can have thousands of files. Reduce the 'Max Items' limit or use a more specific search query to narrow the results.

**I get an error about invalid category.**

Verify the category exists on Commons. Go to https://commons.wikimedia.org/wiki/Category:YourCategoryName to check. Category names must be exact.

**The output has duplicate page titles.**

Each page in a DjVu or PDF file has a unique title (e.g., 'Page:Example.djvu/1'). Duplicates may occur if the same file appears in multiple categories. Use the search query to target specific files.

### FAQ

| Question | Answer |
|---|---|
| What file types does this Actor support? | It supports DjVu and PDF files hosted on Wikimedia Commons. You can choose the file type in the input or scrape by category. |
| Do I need a Wikimedia Commons account or API key? | No. The Actor uses the public Commons API, which requires no authentication. |
| How many pages can I scrape per run? | Free users are limited to 10 pages for preview. Paid users can scrape more pages per run. |
| What is the output format? | Each page is returned as a row with its title and OCR text. You can export the dataset to CSV, JSON, Excel, or XML. |
| Can I scrape all books in a specific category? | Yes. Enter the category name (e.g., 'DjVu\_files' or 'PDF\_files') and the Actor will list all files in that category and extract their text. |
| Can I search for books by keyword? | Yes. Use the search query field to filter files by keyword, such as 'index: filetype:djvu Shakespeare'. |
| Does the Actor extract images or metadata? | It extracts only the OCR text and page title. For file metadata like size or author, use the Commons API directly. |
| What if a page has no OCR text? | The Actor will return an empty string for that page's text. This can happen if the file has no embedded OCR layer. |
| Is the text extracted from DjVu or PDF files accurate? | The text comes from the OCR layer embedded in the file by Commons. Accuracy depends on the original scan and OCR quality. |
| Can I scrape files from multiple categories in one run? | No. You can only specify one category or one search query per run. For multiple categories, run the Actor multiple times. |

### Related actors

Browse the full [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp) for more scrapers.

🆘 **Need help?** Email parseforge@protonmail.com with your run ID, your input, and what you expected.

### Pricing

This Actor uses **pay-per-result** pricing: **$0.004 per result** collected. You are billed only for the results you receive, so a run that returns nothing costs nothing.

⚠️ **Disclaimer.** This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Wikimedia Foundation, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.

# Actor input Schema

## `maxItems` (type: `integer`):

Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000

## `category` (type: `string`):

Wikimedia Commons category to scrape files from (e.g., 'DjVu\_files' or 'PDF\_files'). Leave empty to search by file type.

## `searchQuery` (type: `string`):

Optional search query to filter files (e.g., 'index: filetype:djvu'). Overrides category if provided.

## `fileType` (type: `string`):

Type of files to scrape: 'djvu' or 'pdf'. Used only when category is empty.

## Actor input object example

```json
{
  "maxItems": 10,
  "category": "DjVu_files",
  "fileType": "djvu"
}
```

# Actor output Schema

## `results` (type: `string`):

Complete dataset of all scraped records.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "maxItems": 10,
    "category": "DjVu_files",
    "searchQuery": "",
    "fileType": "djvu"
};

// Run the Actor and wait for it to finish
const run = await client.actor("parseforge/wikimedia-commons-book-text-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "maxItems": 10,
    "category": "DjVu_files",
    "searchQuery": "",
    "fileType": "djvu",
}

# Run the Actor and wait for it to finish
run = client.actor("parseforge/wikimedia-commons-book-text-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "maxItems": 10,
  "category": "DjVu_files",
  "searchQuery": "",
  "fileType": "djvu"
}' |
apify call parseforge/wikimedia-commons-book-text-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/wikimedia-commons-book-text-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7LGrRUSN5h5mPOeQU/builds/D9i00hdtsK84jndED/openapi.json