Wikipedia Scraper: Articles, Summaries & Infoboxes

Wikipedia Articles Scraper pulls articles from Wikipedia through the official REST API, short description, clean summary, full plain-text article, lead image, coordinates, categories and outbound links, in any of the 300+ language editions.

Wikipedia publishes every article through a public API designed for exactly this. This Actor reads it directly: no proxy setup, no browser, no credentials to manage. You get plain text, not wikitext and not stripped HTML.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/wikipedia-articles-scraper.md

What you get

Output field	Meaning
title, pageId, lang	Normalised title, page id and language edition
description	One-line short description (the grey line under the title)
extract	Clean plain-text summary, the article's lead section
fullText	Complete article as plain text (optional)
thumbnail, originalImage	Lead image as { source, width, height }
coordinates	{ lat, lon } for geotagged articles
categories	Visible category memberships, prefix stripped (optional)
links	Outbound article links, main namespace (optional)
lastModified	Timestamp of the revision you received
url	Canonical article URL
scrapedAt	Run timestamp

A RUN_SUMMARY record in the key-value store holds per-run counts, the options used, any article that failed, any article the run limit skipped, and any title that resolved to an article the run already had.

Common use cases

• RAG knowledge bases. Build a licence-clean corpus with includeFullText, chunk it, embed it.
• Entity enrichment. Attach a canonical one-line description and lead image to your own records.
• Multilingual research. Fetch the same title across en, de, fr, ja and diff coverage.
• Geodata. Pull coordinates for thousands of places in one run.
• Link-graph analysis. Map how topics connect via includeLinks.

Quick start

Two articles, summaries only, fast and cheap:

{
 "titles": ["Web scraping", "Berlin"]
}

Full corpus build with metadata:

{
 "titles": ["Web scraping", "Data mining", "https://de.wikipedia.org/wiki/Kartoffel"],
 "languages": ["en"],
 "maxArticles": 500,
 "includeFullText": true,
 "includeLinks": true,
 "includeCategories": true
}

The same topic in four languages:

{
 "titles": ["Artificial intelligence"],
 "languages": ["en", "de", "fr", "ja"]
}

Input

Field	Type	Default	Notes
titles	array	-	Required. Titles (Web scraping), slugs (web_scraping) or full Wikipedia URLs. Mixed input is fine.
languages	array	["en"]	Every plain title is fetched once per language. A title given as a URL already names its wiki and ignores this.
maxArticles	integer	100	Ceiling on the records the whole run delivers, across all titles and languages together. 0 means no limit.
includeFullText	boolean	false	Fetch the complete article as plain text.
includeLinks	boolean	false	Include outbound main-namespace links (first 500).
includeCategories	boolean	false	Include visible categories, Category: prefix stripped.

Two titles × two languages = four records. Redirects are followed automatically, so titles: ["NYC"] returns New York City.

One article reached by several names is one record. berlin, Berlin, Berlin_ and the article URL are the same page, and NYC and New York City land on the same page too — the run keeps the first and lists the rest in RUN_SUMMARY.articlesDuplicate, so you are never charged twice for the same row. The same article in two languages is still two records.

The run limit

maxArticles is a total, not a per-title allowance: 20 titles across 5 languages is 100 records, and maxArticles: 30 returns 30 of them, not 30 each. Articles the ceiling cut off are listed in RUN_SUMMARY.articlesSkipped, so a short dataset is never mistaken for an article Wikipedia does not have — and articles that were never fetched are never charged. A missing article does not consume the budget either; the next title takes its place. Raise the number, or set 0, when you want the whole list. A maxArticles that is negative or not a whole number is rejected before the first request rather than read as "no limit".

Output example

{
 "title": "Berlin",
 "pageId": 3354,
 "lang": "en",
 "description": "Capital and largest city of Germany",
 "extract": "Berlin is the capital and largest city of Germany. With 3.7 million inhabitants, …",
 "fullText": "Berlin is the capital and largest city of Germany…\n\nHistory\n\nEtymology\n\n…",
 "thumbnail": {
 "source": "https://upload.wikimedia.org/wikipedia/commons/thumb/f/f7/Museumsinsel_Berlin_Juli_2021_1_%28cropped%29_b.jpg/330px-…jpg",
 "width": 330,
 "height": 192
 },
 "originalImage": { "source": "https://upload.wikimedia.org/…/Museumsinsel_Berlin_Juli_2021_1_%28cropped%29_b.jpg", "width": 5464, "height": 3186 },
 "coordinates": { "lat": 52.52, "lon": 13.405 },
 "categories": ["Berlin", "Capitals in Europe", "German state capitals"],
 "links": [".berlin", "1. FC Union Berlin", "Aachen"],
 "lastModified": "2026-08-01T21:11:05Z",
 "url": "https://en.wikipedia.org/wiki/Berlin",
 "scrapedAt": "2026-08-06T11:42:00.000Z"
}

fullText, categories and links are omitted entirely unless you switch them on, so the default dataset stays small.

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~wikipedia-articles-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "titles": ["Web scraping", "Berlin"],
 "includeFullText": true
 }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/wikipedia-articles-scraper').call({
 titles: ['Artificial intelligence'],
 languages: ['en', 'de', 'fr'],
 includeFullText: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) console.log(`${a.lang}: ${a.title}, ${a.fullText.length} chars`);

Notes

• Summaries come from the REST API, which is far cleaner than the legacy action API: normalised title, plain-text lead, image and coordinates in a single 2 KB response.
• Full text is one request per article. MediaWiki silently lowers exlimit to 1 for whole-article extracts, so batching is impossible for includeFullText, the Actor does not pretend otherwise. Links and categories are batched, 50 pages per request.
• Extras are fetched by page id, not by title, so redirects and title normalisation cannot mismatch a record with someone else's data.
• Links are capped at 500 per article by the API's own limit; the Actor takes the first page rather than paging forever.
• maxArticles bounds the whole run. It is checked against what the run has actually delivered, so it holds however many titles and languages you combine, and the articles it cut off are named in RUN_SUMMARY.articlesSkipped.
• Duplicates are identified by the page, not by the spelling. Two titles that resolve to the same article in the same language produce one record; the dropped spellings are named in RUN_SUMMARY.articlesDuplicate, and a dropped copy never uses up a place under maxArticles.
• Missing articles don't kill the run. A 404 is recorded in RUN_SUMMARY.failures and the run continues; the Actor only errors out if every article fails.
• Transient errors are retried. 429 and 5xx get four attempts with exponential backoff and jitter.
• Descriptive user agent. Wikimedia throttles generic clients within a handful of requests; every request identifies this Actor.
• HTML is decoded, then stripped, then decoded again. Where a wiki only ships the HTML variant of a summary, entities are resolved before tags are removed so you get real text, not <p>.

FAQ

Do I need a proxy? No. Proxy configuration is not required to run this Actor.

Do I need a Wikipedia account or key? No. You supply no credentials.

Where is the infobox? Wikipedia's infobox is rendered wikitext, not a structured field, the REST API does not expose it. description, coordinates, thumbnail and originalImage cover the high-value infobox facts; for the full structured fact set use the Wikidata Scraper, which is where those facts actually live.

How do I keep a run from getting bigger than I meant? maxArticles caps the records the run delivers in total. It defaults to 100, and 0 lifts it. Because records are titles × languages, that is the number worth setting before a large multilingual run.

What happens if an article is unavailable? It is reported in RUN_SUMMARY.failures and the run continues.

Can I schedule it? Yes, it is designed for scheduled runs. Diff on lastModified to detect edited articles.

Does it follow redirects? Yes. NYC resolves to New York City, and the record carries the resolved title.

Which languages work? Any Wikipedia edition, pass its code (en, de, simple, ja, ar, …).

Can I integrate it with something else? Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.