Wikimedia Commons Book Text Scraper

Wikimedia Commons Book Text Scraper

Scrape book text from Wikimedia Commons DjVu and PDF files. Every page returns its title and file metadata. No login or API key. Export to CSV, JSON, Excel, or XML.

Wikimedia Commons hosts millions of scanned books as DjVu and PDF files, but extracting the text page by page is tedious. This Actor uses the Commons API to list files by category or search, then fetches the OCR text for each page, returning a clean dataset of page titles and their full text content.

Who uses it	What they scrape Wikimedia Commons for
Digital humanities researchers	Extract text from public domain books for corpus analysis or transcription projects.
Data scientists	Build training datasets from scanned book text for NLP models.
Librarians and archivists	Bulk download OCR text from Commons book collections for preservation or indexing.
Language learners and educators	Gather text from historical or foreign-language books for study materials.

What it does

This Actor collects book text from Wikimedia Commons DjVu and PDF files, returning each page as a row with its title and extracted text.

• 📄 Page-level text extraction: Each page of a DjVu or PDF file is returned as a separate row with its full OCR text.
• 📂 Category or search-driven: Scrape all files in a Commons category (e.g., 'DjVu_files') or use a custom search query to target specific books.
• 🔍 File type filter: Choose between DjVu or PDF files when no category is specified.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with Wikimedia Commons data

📚 Build a corpus of public domain books.

A researcher runs the Actor on the 'DjVu_files' category to download OCR text from thousands of scanned books for linguistic analysis.

🔍 Search for specific book content.

A data scientist uses a search query like 'index: filetype:djvu Shakespeare' to extract text only from Shakespeare-related books.

📊 Archive book text for offline access.

A librarian scrapes a Commons category of PDF files to create a local text archive of historical documents.

🤖 Train an OCR correction model.

An NLP engineer collects page text from Commons DjVu files to build a dataset for training a model that improves OCR accuracy.

Why choose this scraper

	What you get
Page text	Full OCR text for each page of the book.
Page title	The title of the page on Commons (e.g., 'Page:Example.djvu/1').
File metadata	File name, category, and search context.

What a Wikimedia Commons record looks like

Every record returns as one flat JSON row. Here is a real one from a run:

{
 "pageid": 40316238,
 "title": "File:Deledda - Cartolina autografa.djvu",
 "fileUrl": "https://commons.wikimedia.org/wiki/File%3ADeledda%20-%20Cartolina%20autografa.djvu",
 "scrapedAt": "2026-09-04T01:20:45.594Z"
}

Every value above comes from a real run. A field a record does not have comes back as null.

Configure the run

Drive the Actor from a Wikimedia Commons category name, a search query, or a file type selection. Filters run as each file is listed, so only matching pages reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10,
 "category": "DjVu_files",
 "fileType": "djvu"
}

A larger pull:

{
 "maxItems": 200,
 "category": "DjVu_files",
 "fileType": "djvu"
}

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect more results per run.

Run it

1. Create a free Apify account.
2. Open the Wikimedia Commons Book Text Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to Wikimedia Commons through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/wikimedia-commons-book-text-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that the category name or search query is correct. Category names are case-sensitive (e.g., 'DjVu_files' not 'djvu_files'). Also ensure the category contains DjVu or PDF files.

The Actor returns empty text for some pages.

Some files on Commons may not have an embedded OCR layer. The Actor will return an empty string for those pages. Try a different file or category.

The run is taking too long.

Large categories can have thousands of files. Reduce the 'Max Items' limit or use a more specific search query to narrow the results.

I get an error about invalid category.

Verify the category exists on Commons. Go to https://commons.wikimedia.org/wiki/Category:YourCategoryName to check. Category names must be exact.

The output has duplicate page titles.

Each page in a DjVu or PDF file has a unique title (e.g., 'Page:Example.djvu/1'). Duplicates may occur if the same file appears in multiple categories. Use the search query to target specific files.

FAQ

Question	Answer
What file types does this Actor support?	It supports DjVu and PDF files hosted on Wikimedia Commons. You can choose the file type in the input or scrape by category.
Do I need a Wikimedia Commons account or API key?	No. The Actor uses the public Commons API, which requires no authentication.
How many pages can I scrape per run?	Free users are limited to 10 pages for preview. Paid users can scrape more pages per run.
What is the output format?	Each page is returned as a row with its title and OCR text. You can export the dataset to CSV, JSON, Excel, or XML.
Can I scrape all books in a specific category?	Yes. Enter the category name (e.g., 'DjVu_files' or 'PDF_files') and the Actor will list all files in that category and extract their text.
Can I search for books by keyword?	Yes. Use the search query field to filter files by keyword, such as 'index: filetype:djvu Shakespeare'.
Does the Actor extract images or metadata?	It extracts only the OCR text and page title. For file metadata like size or author, use the Commons API directly.
What if a page has no OCR text?	The Actor will return an empty string for that page's text. This can happen if the file has no embedded OCR layer.
Is the text extracted from DjVu or PDF files accurate?	The text comes from the OCR layer embedded in the file by Commons. Accuracy depends on the original scan and OCR quality.
Can I scrape files from multiple categories in one run?	No. You can only specify one category or one search query per run. For multiple categories, run the Actor multiple times.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

Pricing

This Actor uses pay-per-result pricing: $0.004 per result collected. You are billed only for the results you receive, so a run that returns nothing costs nothing.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Wikimedia Foundation, Inc. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.