# Wikipedia Scraper: Articles, Summaries & Infoboxes (`arman-bd/wikipedia-articles-scraper`) Actor

Scrape Wikipedia through the official REST API: summaries, full extracts, images, coordinates, categories and links. 300+ languages, no key, no proxies.

- **URL**: https://apify.com/arman-bd/wikipedia-articles-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** Automation, AI, MCP servers
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.37 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Wikipedia Scraper: Articles, Summaries & Infoboxes

![Wikipedia Articles Scraper: Short descriptions, clean summaries, full text, images, coordinates, categories and links from any Wikipedia edition](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/wikipedia-articles-scraper.jpg)

**Wikipedia Articles Scraper** pulls articles from **Wikipedia** through the official REST API, short description, clean summary, full plain-text article, lead image, coordinates, categories and outbound links, in any of the 300+ language editions.

Wikipedia publishes every article through a public API designed for exactly this. This Actor reads it directly: **no proxy setup, no browser, no credentials to manage.** You get plain text, not wikitext and not stripped HTML.

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/wikipedia-articles-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/wikipedia-articles-scraper.md
```

### What you get

| Output field | Meaning |
|---|---|
| `title`, `pageId`, `lang` | Normalised title, page id and language edition |
| `description` | One-line short description (the grey line under the title) |
| `extract` | Clean plain-text summary, the article's lead section |
| `fullText` | Complete article as plain text (optional) |
| `thumbnail`, `originalImage` | Lead image as `{ source, width, height }` |
| `coordinates` | `{ lat, lon }` for geotagged articles |
| `categories` | Visible category memberships, prefix stripped (optional) |
| `links` | Outbound article links, main namespace (optional) |
| `lastModified` | Timestamp of the revision you received |
| `url` | Canonical article URL |
| `scrapedAt` | Run timestamp |

A `RUN_SUMMARY` record in the key-value store holds per-run counts, the options used, any article that failed, any article the run limit skipped, and any title that resolved to an article the run already had.

### Common use cases

- **RAG knowledge bases.** Build a licence-clean corpus with `includeFullText`, chunk it, embed it.
- **Entity enrichment.** Attach a canonical one-line description and lead image to your own records.
- **Multilingual research.** Fetch the same title across `en`, `de`, `fr`, `ja` and diff coverage.
- **Geodata.** Pull coordinates for thousands of places in one run.
- **Link-graph analysis.** Map how topics connect via `includeLinks`.

### Quick start

Two articles, summaries only, fast and cheap:

```json
{
 "titles": ["Web scraping", "Berlin"]
}
```

Full corpus build with metadata:

```json
{
 "titles": ["Web scraping", "Data mining", "https://de.wikipedia.org/wiki/Kartoffel"],
 "languages": ["en"],
 "maxArticles": 500,
 "includeFullText": true,
 "includeLinks": true,
 "includeCategories": true
}
```

The same topic in four languages:

```json
{
 "titles": ["Artificial intelligence"],
 "languages": ["en", "de", "fr", "ja"]
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `titles` | array | - | **Required.** Titles (`Web scraping`), slugs (`web_scraping`) or full Wikipedia URLs. Mixed input is fine. |
| `languages` | array | `["en"]` | Every *plain* title is fetched once per language. A title given as a URL already names its wiki and ignores this. |
| `maxArticles` | integer | `100` | Ceiling on the records the **whole run** delivers, across all titles and languages together. `0` means no limit. |
| `includeFullText` | boolean | `false` | Fetch the complete article as plain text. |
| `includeLinks` | boolean | `false` | Include outbound main-namespace links (first 500). |
| `includeCategories` | boolean | `false` | Include visible categories, `Category:` prefix stripped. |

Two titles × two languages = four records. Redirects are followed automatically, so `titles: ["NYC"]` returns *New York City*.

One article reached by several names is one record. `berlin`, `Berlin`, `Berlin_` and the article URL are the same page, and `NYC` and `New York City` land on the same page too — the run keeps the first and lists the rest in `RUN_SUMMARY.articlesDuplicate`, so you are never charged twice for the same row. The same article in two languages is still two records.

#### The run limit

`maxArticles` is a total, not a per-title allowance: 20 titles across 5 languages is 100 records, and `maxArticles: 30` returns 30 of them, not 30 each. Articles the ceiling cut off are listed in `RUN_SUMMARY.articlesSkipped`, so a short dataset is never mistaken for an article Wikipedia does not have — and articles that were never fetched are never charged. A missing article does not consume the budget either; the next title takes its place. Raise the number, or set `0`, when you want the whole list. A `maxArticles` that is negative or not a whole number is rejected before the first request rather than read as "no limit".

### Output example

```json
{
 "title": "Berlin",
 "pageId": 3354,
 "lang": "en",
 "description": "Capital and largest city of Germany",
 "extract": "Berlin is the capital and largest city of Germany. With 3.7 million inhabitants, …",
 "fullText": "Berlin is the capital and largest city of Germany…\n\nHistory\n\nEtymology\n\n…",
 "thumbnail": {
 "source": "https://upload.wikimedia.org/wikipedia/commons/thumb/f/f7/Museumsinsel_Berlin_Juli_2021_1_%28cropped%29_b.jpg/330px-…jpg",
 "width": 330,
 "height": 192
 },
 "originalImage": { "source": "https://upload.wikimedia.org/…/Museumsinsel_Berlin_Juli_2021_1_%28cropped%29_b.jpg", "width": 5464, "height": 3186 },
 "coordinates": { "lat": 52.52, "lon": 13.405 },
 "categories": ["Berlin", "Capitals in Europe", "German state capitals"],
 "links": [".berlin", "1. FC Union Berlin", "Aachen"],
 "lastModified": "2026-08-01T21:11:05Z",
 "url": "https://en.wikipedia.org/wiki/Berlin",
 "scrapedAt": "2026-08-06T11:42:00.000Z"
}
```

`fullText`, `categories` and `links` are omitted entirely unless you switch them on, so the default dataset stays small.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~wikipedia-articles-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "titles": ["Web scraping", "Berlin"],
 "includeFullText": true
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/wikipedia-articles-scraper').call({
 titles: ['Artificial intelligence'],
 languages: ['en', 'de', 'fr'],
 includeFullText: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) console.log(`${a.lang}: ${a.title}, ${a.fullText.length} chars`);
```

### Notes

- **Summaries come from the REST API**, which is far cleaner than the legacy action API: normalised title, plain-text lead, image and coordinates in a single 2 KB response.
- **Full text is one request per article.** MediaWiki silently lowers `exlimit` to 1 for whole-article extracts, so batching is impossible for `includeFullText`, the Actor does not pretend otherwise. Links and categories *are* batched, 50 pages per request.
- **Extras are fetched by page id, not by title**, so redirects and title normalisation cannot mismatch a record with someone else's data.
- **Links are capped at 500** per article by the API's own limit; the Actor takes the first page rather than paging forever.
- **`maxArticles` bounds the whole run.** It is checked against what the run has actually delivered, so it holds however many titles and languages you combine, and the articles it cut off are named in `RUN_SUMMARY.articlesSkipped`.
- **Duplicates are identified by the page, not by the spelling.** Two titles that resolve to the same article in the same language produce one record; the dropped spellings are named in `RUN_SUMMARY.articlesDuplicate`, and a dropped copy never uses up a place under `maxArticles`.
- **Missing articles don't kill the run.** A 404 is recorded in `RUN_SUMMARY.failures` and the run continues; the Actor only errors out if *every* article fails.
- **Transient errors are retried.** 429 and 5xx get four attempts with exponential backoff and jitter.
- **Descriptive user agent.** Wikimedia throttles generic clients within a handful of requests; every request identifies this Actor.
- **HTML is decoded, then stripped, then decoded again.** Where a wiki only ships the HTML variant of a summary, entities are resolved before tags are removed so you get real text, not `<p>`.

### FAQ

**Do I need a proxy?** No. Proxy configuration is not required to run this Actor.

**Do I need a Wikipedia account or key?** No. You supply no credentials.

**Where is the infobox?** Wikipedia's infobox is rendered wikitext, not a structured field, the REST API does not expose it. `description`, `coordinates`, `thumbnail` and `originalImage` cover the high-value infobox facts; for the full structured fact set use the Wikidata Scraper, which is where those facts actually live.

**How do I keep a run from getting bigger than I meant?** `maxArticles` caps the records the run delivers in total. It defaults to 100, and `0` lifts it. Because records are titles × languages, that is the number worth setting before a large multilingual run.

**What happens if an article is unavailable?** It is reported in `RUN_SUMMARY.failures` and the run continues.

**Can I schedule it?** Yes, it is designed for scheduled runs. Diff on `lastModified` to detect edited articles.

**Does it follow redirects?** Yes. `NYC` resolves to *New York City*, and the record carries the resolved title.

**Which languages work?** Any Wikipedia edition, pass its code (`en`, `de`, `simple`, `ja`, `ar`, …).

**Can I integrate it with something else?** Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.

# Actor input Schema

## `titles` (type: `array`):

Article titles ('Web scraping'), slugs ('web\_scraping') or full Wikipedia URLs. A URL carries its own language and is fetched only from that wiki. Two entries that resolve to the same article in the same language are delivered, and charged, once.

## `languages` (type: `array`):

Language codes to fetch each plain title from. 'en', 'de', 'fr'. Every title is fetched once per language, so two titles and two languages produce four records.

## `maxArticles` (type: `integer`):

Ceiling on the records the whole run delivers, counted across every title and language together, not per title. Records are titles x languages, so a short list can still be a large run. Set 0 for no limit.

## `includeFullText` (type: `boolean`):

Fetch the complete article as plain text. MediaWiki caps full extracts at one article per request, so this makes the run one request per article slower.

## `includeLinks` (type: `boolean`):

Include the article titles this page links to (main namespace only, first 500).

## `includeCategories` (type: `boolean`):

Include the visible category memberships of the article, with the 'Category:' prefix stripped.

## Actor input object example

```json
{
  "titles": [
    "Web scraping",
    "https://de.wikipedia.org/wiki/Berlin"
  ],
  "languages": [
    "en",
    "de"
  ],
  "maxArticles": 100,
  "includeFullText": false,
  "includeLinks": false,
  "includeCategories": false
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "titles": [
        "Web scraping",
        "Berlin"
    ],
    "languages": [
        "en"
    ],
    "maxArticles": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/wikipedia-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "titles": [
        "Web scraping",
        "Berlin",
    ],
    "languages": ["en"],
    "maxArticles": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/wikipedia-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "titles": [
    "Web scraping",
    "Berlin"
  ],
  "languages": [
    "en"
  ],
  "maxArticles": 100
}' |
apify call arman-bd/wikipedia-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/wikipedia-articles-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/jJflXh85efJpOZLYo/builds/DmZc74iiTM7daACGo/openapi.json