# Wikipedia Scraper: Articles, Search & Pageviews (`tidyharvest/wikipedia-scraper`) Actor

Run many Wikipedia searches at once in any language, or pull articles by title, link or category, or the day's most-viewed: summary, full text, categories, Wikidata ID, coordinates and daily pageviews. Official APIs, no proxy. $0.90 per 1,000 articles.

- **URL**: https://apify.com/tidyharvest/wikipedia-scraper.md
- **Developed by:** [TidyHarvest](https://apify.com/tidyharvest) (community)
- **Categories:** AI, Education
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.54 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Wikipedia Scraper: Articles, Search & Pageviews

Get **Wikipedia articles** as clean structured data: **title, short description, lead summary, optional full plain text, categories, coordinates, main image, Wikidata ID, last edit and revision, other-language versions and daily pageviews**. Run one or many searches in any of the 300+ Wikipedia languages, pass a list of titles or links, pull a whole **category**, or get the **most-viewed articles** of a day.

It reads only the **official Wikimedia APIs** (the MediaWiki Action API and the Wikimedia Pageviews API), politely and one request at a time. **No browser, no proxy, no login or API key.** A default run of 25 articles takes a few seconds. **$0.90 per 1,000 articles**, with pageviews, full text and language links included in that price.

### What you can do with it

- **AI and RAG datasets:** clean plain-text lead summaries or full articles, with license and source URL on every row, ready for embeddings or fine-tuning corpora.
- **Knowledge graphs and entity enrichment:** map names to Wikidata IDs, short descriptions, coordinates and categories.
- **Trend and attention research:** daily pageviews per article, or the top articles of any day in any language.
- **SEO and content research:** see which topics are covered, how long articles are and how much traffic they get.
- **Education and research:** pull every article in a category (and its subcategories) for a reading list or study.
- **Monitoring:** track `revisionId` / `lastEdited` to detect edits to pages you care about.

### Input

| Field | What it does |
|---|---|
| `mode` | `search`, `titles`, `category` or `top` (most-viewed of a day). Default `search`. |
| `queries` | Search texts for `search` mode, one per line; each runs its own search. Empty = `web scraping`. |
| `maxResultsPerQuery` | Cap per search when you give several (empty = only `maxItems` applies). |
| `titles` | Titles or links for `titles` mode, in any mix of languages (`https://de.wikipedia.org/wiki/Berlin`, mobile links, `?curid=` links). Redirects are followed; titles that don't exist are listed in the run summary and not charged. |
| `category` | Category name or link for `category` mode; `categoryDepth: 1` adds direct subcategories. |
| `date` | Day for `top` mode (`YYYY-MM-DD`); empty = latest available. |
| `language` | Wikipedia edition (`en`, `de`, `fr`, `ja`, `simple`, ...). Default `en`. |
| `maxItems` | Hard cap on articles for the whole run (and cost). Default 25, max 5,000. |
| `includeFullText` | Add the whole article as plain text and an exact word count. |
| `includePageviews`, `pageviewDays` | Add total and daily human pageviews for the last N days (default 30). |
| `includeLanguageLinks` | Add the same article's title and URL in every other language. |

Examples:

```json
{ "queries": ["large language model", "retrieval-augmented generation", "vector database"], "maxResultsPerQuery": 20, "maxItems": 60, "includeFullText": true }
```

```json
{ "mode": "category", "category": "Machine learning", "categoryDepth": 1, "maxItems": 200, "includePageviews": true }
```

### Output

One item per article:

```json
{
  "title": "Web scraping",
  "pageId": 2696619,
  "lang": "en",
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "description": "Method of extracting data from websites",
  "summary": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. ...",
  "wordCount": 4123,
  "lengthBytes": 34836,
  "categories": ["Web scraping"],
  "latitude": null, "longitude": null,
  "imageUrl": null, "thumbnailUrl": null,
  "wikidataId": "Q665452",
  "isDisambiguation": false,
  "lastEdited": "2026-09-19T09:13:04Z",
  "revisionId": 1375681792,
  "searchRank": 1,
  "pageviewsTotal": 58677,
  "pageviewsFrom": "2026-09-12", "pageviewsTo": "2026-10-11",
  "pageviewsDaily": [{"date": "2026-09-12", "views": 2042}],
  "license": "CC BY-SA 4.0",
  "licenseUrl": "https://creativecommons.org/licenses/by-sa/4.0/",
  "attribution": "Wikipedia contributors",
  "scrapedAt": "2026-10-12T08:00:00+00:00"
}
```

`fullText` appears when `includeFullText` is on; `languagesCount` and `languageLinks` when `includeLanguageLinks` is on; `searchQuery` and `searchRank` in `search` mode (an article found by two queries is returned once); `topDate`, `topRank`, `topViews` in `top` mode (the main page and special pages are skipped); `category` in `category` mode. `wordCount` comes from search results or from the full text (for Chinese, Japanese, Thai and other languages written without spaces it counts characters; `charCount` is the full text's length); in other modes without full text it is `null` (use `lengthBytes`). Images are the article's free-licensed lead image only.

### How to use

1. Pick a mode and fill in the query, titles or category (or just press Start to try the default search).
2. Set `language` and `maxItems`, and switch on full text or pageviews if you need them.
3. Run, then download the dataset as JSON, CSV or Excel, or connect it to Sheets, Make, Zapier or your own code through the Apify API.

### Pricing

Pay per result: **$0.90 per 1,000 articles** (`article-scraped`), no start fee. Pageviews, full text and language links are part of the article row and cost nothing extra. `maxItems` caps the run before you are charged. A run that finds nothing (unknown category, no search hits, bad language code) fails with a clear message instead of returning an empty dataset.

### Reusing Wikipedia content (license)

Wikipedia text is licensed under **Creative Commons Attribution-ShareAlike 4.0 (CC BY-SA 4.0)**. If you republish it, you must:

- **credit** it, e.g. "From Wikipedia, the free encyclopedia, by Wikipedia contributors" with a link to the article (`url`, which also leads to the full author history);
- **link the license** (`licenseUrl`) and say if you changed the text;
- **share alike:** text you build from it and redistribute must carry the same license.

Every row includes `url`, `license`, `licenseUrl` and `attribution` so you can do this automatically. Images can carry their own, different licenses; check the file page before reusing one. Facts, pageview counts and Wikidata IDs are not covered by the text license (Wikidata is CC0). This is general information, not legal advice.

### Limits and notes

- Search, category and title lookups cover the article namespace only (no talk, user or file pages).
- Category mode lists direct members (plus one level of subcategories with `categoryDepth: 1`), in Wikipedia's sort order.
- Pageviews are human (non-bot) views across desktop and mobile, available from July 2015 and usually up to yesterday (UTC).
- The actor follows Wikimedia's API etiquette (descriptive User-Agent, serial requests, `maxlag`), so very large runs with full text and pageviews take a few minutes (two extra requests per article).
- For bulk mirroring of all of Wikipedia, use the official database dumps or Wikimedia Enterprise instead of this actor.
- Independent tool, **not affiliated with or endorsed by Wikipedia or the Wikimedia Foundation.** "Wikipedia" is a trademark of the Wikimedia Foundation.

# Actor input Schema

## `mode` (type: `string`):

`search` = full-text search for each of `queries`; `titles` = the articles listed in `titles`; `category` = articles in `category`; `top` = the most-viewed articles of one day.

## `queries` (type: `array`):

Used in `search` mode. One search per line, same full-text search as the Wikipedia search box, results in relevance order. Each row says which query found it; an article found by two queries is returned once.

## `query` (type: `string`):

Same as one line in `queries`; kept for existing integrations.

## `maxResultsPerQuery` (type: `integer`):

Used in `search` mode with several queries: cap for each query. Empty = no per-query cap (only Max articles applies).

## `titles` (type: `array`):

Used in `titles` mode. One per line: an article title (`Alan Turing`) or a full link (`https://de.wikipedia.org/wiki/Berlin`; the link's language wins). Redirects are followed.

## `category` (type: `string`):

Used in `category` mode. Category name with or without the `Category:` prefix (`Machine learning`), or a category link.

## `categoryDepth` (type: `integer`):

0 = only articles directly in the category; 1 = also articles in its direct subcategories.

## `date` (type: `string`):

Used in `top` mode. Leave empty for the latest available day (usually yesterday, UTC).

## `language` (type: `string`):

Language edition code: `en`, `de`, `fr`, `es`, `ja`, `zh`, `simple`, ... (the part before `.wikipedia.org`).

## `maxItems` (type: `integer`):

Hard cap on articles returned in this run, across all queries or titles (also caps your cost).

## `includeFullText` (type: `boolean`):

Adds `fullText` (the whole article as plain text, no markup) and an exact `wordCount`. One extra request per article, so slower.

## `includePageviews` (type: `boolean`):

Adds `pageviewsTotal` and a `pageviewsDaily` series (human views, all platforms) for the last N days. Included in the article price.

## `pageviewDays` (type: `integer`):

How many days of pageviews, ending yesterday (UTC).

## `includeLanguageLinks` (type: `boolean`):

Adds `languagesCount` and `languageLinks` (title and URL of the same article in every other language).

## Actor input object example

```json
{
  "mode": "search",
  "queries": [
    "web scraping"
  ],
  "titles": [
    "Alan Turing",
    "https://en.wikipedia.org/wiki/Large_language_model"
  ],
  "category": "Machine learning",
  "categoryDepth": 0,
  "language": "en",
  "maxItems": 25,
  "includeFullText": false,
  "includePageviews": false,
  "pageviewDays": 30,
  "includeLanguageLinks": false
}
```

# Actor output Schema

## `articles` (type: `string`):

Wikipedia articles, one item per article (CC BY-SA 4.0, attribution: Wikipedia contributors).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "queries": [
        "web scraping"
    ],
    "titles": [
        "Alan Turing",
        "https://en.wikipedia.org/wiki/Large_language_model"
    ],
    "category": "Machine learning"
};

// Run the Actor and wait for it to finish
const run = await client.actor("tidyharvest/wikipedia-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "queries": ["web scraping"],
    "titles": [
        "Alan Turing",
        "https://en.wikipedia.org/wiki/Large_language_model",
    ],
    "category": "Machine learning",
}

# Run the Actor and wait for it to finish
run = client.actor("tidyharvest/wikipedia-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "queries": [
    "web scraping"
  ],
  "titles": [
    "Alan Turing",
    "https://en.wikipedia.org/wiki/Large_language_model"
  ],
  "category": "Machine learning"
}' |
apify call tidyharvest/wikipedia-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,tidyharvest/wikipedia-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/bstKxLn8fVSNbTgcd/builds/yy8clFAxNk1Qj2Wbg/openapi.json