Wikipedia Scraper · Articles, Extracts, Categories & Metadata

The Wikipedia Scraper is a high-performance HTTP API integration for extracting Wikipedia articles, lead summary extracts, full text metadata, article categories, and page revision info across 300+ Wikipedia language editions. Designed for researchers, AI data pipelines, RAG systems, and knowledge graph construction.

Maintained by reapX. Every row cites the Apify run that produced it — nothing is inferred, modelled or filled in, and a field absent from the source is absent from the row. The extracted archive for this source is browsable at reapx.dev/data/wikipedia-scraper/ and mirrored as an open dataset on Hugging Face and Kaggle. Questions: reapxdev@proton.me

Why use a Wikipedia Scraper?

Wikipedia is the world's largest open encyclopedia, holding millions of structured and semi-structured articles. Extracting data directly via standard scrapers often results in rate limits or complex HTML parsing. This scraper provides direct, structured JSON output for:

• LLM Training & Fine-Tuning: Extract clean text summaries and topic-specific corpora.
• RAG & Knowledge Bases: Power retrieval augmented generation pipelines with curated encyclopedia extracts.
• Academic Research: Gather multilingual Wikipedia datasets across topics and categories.
• Entity Resolution & Categorization: Map Wikipedia category trees and canonical article URLs.

Features

• Multilingual Support: Query any Wikipedia language edition (en, de, fr, es, ja, ar, etc.).
• Topic Search & Category Extraction: Search by keywords or extract entire Wikipedia category trees.
• Clean Text Extracts: Extract plain text intro summaries without HTML markup bloat.
• Metadata & Categories: Retrieve content length, revision timestamps, categories, and page IDs.
• Pay-Per-Event Pricing: Pay only per successful article row returned ($0.001/row with volume discounts). Empty searches and blocked requests cost $0.

How to Scrape Wikipedia Articles

1. Set Search Terms or Category: Provide search queries (e.g. ["Artificial intelligence", "Quantum computing"]) or a category (e.g. Category:Artificial intelligence).
2. Select Language: Pick the target language code (en, de, fr, es, etc.).
3. Configure Limits: Choose maximum number of articles to return (1-500).
4. Run Actor: Receive structured JSON output instantly.

⬇️ Input

Every field is described here exactly as the Actor reads it. Run it with the prefilled input and it returns real rows without any further configuration.

Field	Type	Required	Prefilled	What it does
searchTerms	array	yes	["Artificial intelligence", "Quant	One search query per line. Each is run against Wikipedia's own search index (e.g. Artificial intelligence), and results from every term are merged and de-duplicated by page id.
category	string	no	-	Extract the members of one Wikipedia category, with or without the prefix (e.g. Category:Machine learning or Machine learning). Category members are collected first, then search terms top...
language	string	yes	"en"	Language edition to read, as the subdomain code (e.g. en, de, fr, es, ja, ar). The Actor calls https://<code>.wikipedia.org/w/api.php, so any edition that exists is reachable.
namespace	string	no	-	Restrict results to one MediaWiki namespace id (e.g. 0 for articles, 14 for category pages, 100 for portals).
searchSort	string	no	-	Order Wikipedia's search index returns matches in. relevance is the search default; last_edit_desc gives you the most recently edited matches, incoming_links_desc the most linked-to,...
maxArticles	integer	yes	50	Hard cap on rows for the run, across every search term and the category together (1 to 500).
includeExtract	boolean	no	true	Return the article's plain-text lead section in extract. Text is requested 20 pages per call, which is MediaWiki's exlimit ceiling, so every row carries its extract and not just the first...
extractScope	string	no	"intro"	How much text extract carries: intro is the lead section only (MediaWiki exintro), full is the whole article as plain text.
extractChars	integer	no	-	Truncate extract to roughly this many characters, at a word boundary (MediaWiki exchars, maximum 1200).
includeCategories	boolean	no	true	Return every category the page belongs to in categories, continued until the list is complete rather than cut off at the first 500 of the batch.
includeHiddenCategories	boolean	no	false	Include Wikipedia's hidden housekeeping categories, such as All articles with unsourced statements or Articles with short description.
includeWikidataId	boolean	no	true	Return the page's Wikidata item in wikidataId (e.g. Q11660), read from pageprops.wikibase_item in the same request as the text.
includePageImage	boolean	no	true	Return a 400px thumbnail URL for the page's lead image in thumbnailUrl.
includeLanguageCount	boolean	no	false	Count how many other language editions carry the same page and return it in languageCount - a good proxy for how significant a topic is.
includePageviews	boolean	no	true	Return how many times the page was viewed over the last 30 days in pageviews30d, summed from MediaWiki's own daily prop=pageviews series.
includeLinks	boolean	no	false	Return the article-namespace pages this page links to, as links, with the total in linkCount. Template, file and category links are excluded, so what you get is the topical link graph.

What empty means, and what a larger value costs

The two questions an input table normally does not answer. Both columns are read off this Actor's own code and its own price list.

Field	Left empty	Turned up
searchTerms	Leave empty when you are using Wikipedia Category instead; with neither set the Actor searches Artificial intelligence.	More terms means more requests and more rows, and you pay per row returned.
category	Leave empty to run search terms only.	A category walks the category listing 500 pages per request, which is cheaper per row than search.
language	Falls back to en.	The language also lands on every row, so multi-language corpora stay separable.
namespace	Leave empty to accept the API's own default, which is the article namespace for search.	A non-article namespace returns pages whose length and extract look nothing like an encyclopedia article, so keep it empty unless you want those.
searchSort	Leave empty for the API's own relevance order.	Ordering changes which rows you get when your limit is smaller than the match count, not how many you are charged for. Applies to search terms only, never to a category.
maxArticles	Falls back to 50.	Each row returned is one $0.001 charge, so this field is your cost cap as well as your row cap. At the ceiling of 500 rows that is $0.50 at list price.
includeExtract	Extract is an empty string and the run makes fewer, smaller requests.	Extracts dominate the size of each row, not the price - you are charged per row either way.
extractScope	Falls back to intro.	Full can return tens of kilobytes per row, which makes the dataset far heavier to move and to embed; the price per row does not change.
extractChars	No truncation - you get the whole lead section, or the whole article when scope is full.	A small limit keeps rows uniform and cheap to embed for RAG; it does not change the number of rows or the price.
includeCategories	Categories is an empty array.	A page with hundreds of categories can need a follow-up request, so leaving this on can add requests but never adds rows or cost.
includeHiddenCategories	Hidden categories are filtered out at the API with clshow=!hidden, so categories holds only the topical ones a reader sees.	Turning it on typically doubles the length of the category list without adding topical signal.
includeWikidataId	WikidataId is an empty string.	Free - no extra request and no extra charge - and it is what lets you join these rows to Wikidata, DBpedia or your own entity table across languages.
includePageImage	ThumbnailUrl is an empty string.	Free to request, but many articles have no lead image at all, so expect this field to be populated on a minority of rows and never treat it as required.
includeLanguageCount	LanguageCount is 0.	Language links are paged separately by the API, so turning this on can add a continuation request per batch of 20 pages. It adds no rows and no cost.
includePageviews	Pageviews30d is 0.	Free - it rides along in the same request as the text - and it is the only demand signal Wikipedia exposes, so it is what lets you rank a topic corpus by real readership rather...
includeLinks	Links is an empty array and linkCount is 0.	This is the one switch that really costs requests - links page 500 at a time across the whole batch, so a 25-row run measured 37 requests with it on against 4 with it off. It adds...

⬆️ Output

One row per record, as JSON, CSV, Excel or XML, and through the API. Every field below is on every row this Actor writes; a field the source did not return is empty rather than guessed.

Field	Type	What it gives you	Example
id	string	Numeric MediaWiki page id as a string. Stable across edits and renames, and the key every other Wikipedia interface...	"1164"
title	string	Current title of the page in the language edition that was read.	"Artificial intelligence"
url	string	Canonical URL of the page, as returned by the API rather than assembled from the title.	"https://en.wikipedia.org/wiki/Artificial_intelligence"
namespace	integer	MediaWiki namespace id. 0 is an article, 14 a category page, 100 a portal.	0
extract	string	Lead section as plain text, or the whole article when Extract Scope is full. Empty only when Include Lead Extract is...	"Artificial intelligence (AI) is the capability of comput...
language	string	Language edition the row was read from, matching the subdomain that served it.	"en"
categories	array	Every category the page belongs to, hidden maintenance categories excluded unless you asked for them.	["Artificial intelligence", "Computational neuroscience"]
wikidataId	string	Wikidata item for this page, which is the join key to the same entity in every other language edition.	"Q11660"
thumbnailUrl	string	400px thumbnail of the page's lead image. Empty on the many articles that have no lead image.	"https://upload.wikimedia.org/wikipedia/commons/thumb/e/e...
languageCount	integer	How many other language editions carry this page. Zero unless Include Language Edition Count is on.	132
length	integer	Size of the page wikitext in bytes, as MediaWiki reports it.	154200
revisionId	integer	Id of the revision that was current when the row was read. Quote it and anyone can fetch the exact text you saw.	1301944238
lastModified	string	ISO timestamp of the last change MediaWiki recorded for the page.	"2026-07-23T12:04:46Z"
scrapedAt	string	ISO timestamp of the run that produced the row.	"2026-08-04T10:49:52+00:00"
pageviews30d	integer	Total human and automated pageviews over the trailing 30 days, summed from MediaWiki's daily series. Zero when Include...	291333
linkCount	integer	How many article-namespace pages this page links to. Zero unless Include Outgoing Article Links is on.	1981
links	array	Titles of the article-namespace pages this page links to. Empty unless Include Outgoing Article Links is on.	["Deep learning", "Neural network"]

Dataset views

The dataset ships with named views, so the Output tab, the API and any agent reading this Actor get the columns that matter for a job without asking for all of them.

Overview (?view=overview) - 8 columns: id, title, url, language, namespace, pageviews30d, length, lastModified

Column	Header
id	Page ID
title	Title
url	URL
language	Language
namespace	Namespace
pageviews30d	Views (30d)
length	Size (bytes)
lastModified	Last Modified

Text & Categories (?view=content) - 6 columns: id, title, extract, categories, length, url

Column	Header
id	Page ID
title	Title
extract	Extract
categories	Categories
length	Size (bytes)
url	URL

Entity IDs & Provenance (?view=provenance) - 6 columns: id, title, wikidataId, revisionId, lastModified, scrapedAt

Column	Header
id	Page ID
title	Title
wikidataId	Wikidata ID
revisionId	Revision ID
lastModified	Last Modified
scrapedAt	Scraped At

Fetch one directly:

GET https://api.apify.com/v2/datasets/{datasetId}/items?view=overview&format=csv

One real row

Unedited, from this Actor's own most recent successful run.

{
  "id": "13942135",
  "title": "Jonas Cicenas",
  "url": "https://de.wikipedia.org/wiki/Jonas_Cicenas",
  "namespace": 0,
  "extract": "Jonas Cicėnas (* 17. Dezember 1974 in Klaipėda) ist ein litauischer Biochemiker, Bioinformatiker und Biokurator. Er ist Experte für Kinasen am Schweizerischen Institut für Bioinformatik (SIB) und Dozent für Programmierung.",
  "language": "de",
  "categories": [
    "Kategorie:Biochemiker",
    "Kategorie:Bioinformatiker",
    "Kategorie:Geboren 1974",
    "Kategorie:Litauer",
    "Kategorie:Mann"
  ],
  "wikidataId": "",
  "thumbnailUrl": "",
  "languageCount": 1,
  "length": 4326,
  "revisionId": 269367798,
  "lastModified": "2026-08-04T11:23:05Z",
  "scrapedAt": "2026-08-04T11:25:34+00:00"
}

🧪 Worked examples

Every block below is an input this Actor has actually run, with the row count of the run that produced it. Copy one straight into the Console.

One category, full article text, Wikidata ids attached

{
  "searchTerms": [
    "Machine learning"
  ],
  "language": "en",
  "category": "Category:Machine learning",
  "maxArticles": 60,
  "extractScope": "full",
  "extractChars": 900,
  "includeWikidataId": true
}

That run returned 60 rows in 13s.

German edition, most recently edited first, with language-edition counts

{
  "searchTerms": [
    "Künstliche Intelligenz",
    "Maschinelles Lernen"
  ],
  "language": "de",
  "maxArticles": 40,
  "searchSort": "last_edit_desc",
  "includeLanguageCount": true
}

That run returned 40 rows in 9s.

Wikipedia Scraper · Artificial Intelligence & Machine Learning

Extract English Wikipedia articles and extracts for AI, machine learning, neural networks, and deep learning.

{
  "searchTerms": [
    "Artificial intelligence",
    "Machine learning",
    "Deep learning",
    "Neural network",
    "Large language model"
  ],
  "language": "en",
  "maxArticles": 50
}

That run returned 50 rows.

Wikipedia Scraper · Quantum Computing & Physics

Extract articles covering quantum computing, qubits, quantum mechanics, and superposition.

{
  "searchTerms": [
    "Quantum computing",
    "Qubit",
    "Quantum mechanics",
    "Quantum entanglement",
    "Quantum supremacy"
  ],
  "language": "en",
  "maxArticles": 50
}

That run returned 50 rows.

Wikipedia Scraper · French Quantum Physics

Extract French Wikipedia articles on quantum mechanics, computing, and physics.

{
  "searchTerms": [
    "Physique quantique",
    "Mécanique quantique",
    "Informatique quantique"
  ],
  "language": "fr",
  "maxArticles": 50
}

That run returned 50 rows.

Wikipedia Scraper · Macroeconomics & Global Finance

Extract Wikipedia articles on monetary policy, central banking, inflation, and global financial markets.

{
  "searchTerms": [
    "Macroeconomics",
    "Monetary policy",
    "Central bank",
    "Inflation",
    "Financial market"
  ],
  "language": "en",
  "maxArticles": 50
}

That run returned 50 rows.

How it works

The endpoint it calls

Plain HTTP against the source's own public API. No headless browser, no login, no key of yours:

https://<language>.wikipedia.org/w/api.php

One host per language edition and nothing else: the edition code you pass becomes the subdomain, so a German run never touches the English index.

The paging it does

Identity first, text second, because the Action API pages those two things differently.

1. A generator walks the result set - generator=search at up to 50 pages per request, generator=categorymembers at up to 500 - and the API's own continue token is followed until your limit is reached or the source runs out. prop=info rides along, so page id, title, canonical URL, namespace, byte length, last revision id and last-touched timestamp all arrive without a second request.
2. Those page ids are then filled in 20 at a time with prop=extracts|categories|pageprops|pageimages. Twenty is MediaWiki's exlimit ceiling, and its default is 1 - which is why asking for text in the same call as the search returns an extract for the first article and an empty string for every one after it. Categories are continued the same way, because cllimit is a budget for the whole batch rather than a per-page limit.

Two real costs follow. A 50-row run with intro extracts is about 4 requests end to end. Setting Extract Scope to full makes MediaWiki serve one page per request whatever exlimit asks for, so a 60-row full-text run measured 61 requests and 13s against 9s for the same size at intro scope.

The rate it holds

Every request is retried up to 3 times, with the wait doubling between attempts, and HTTP 403, 429 are treated as retryable rather than fatal. Each request times out at 20s.

Requests are serial, never a burst, and every one carries a descriptive User-Agent with a contact address, which is what the Wikimedia User-Agent policy asks for. No API key, no account, no proxy.

What you are charged

One article-returned event per row written to the dataset, $0.001 at list price. Charged once per complete Wikipedia article row written to the default dataset. Blocked sources, empty result sets and articles filtered out are never charged.

Apify plan tier	Price per row
Free	$0.001
Bronze	$0.0009
Silver	$0.0008
Gold	$0.0007
Platinum	$0.0006
Diamond	$0.0005

Apify platform usage is absorbed, not passed on, so the row price is the whole bill.

Why the dataset is correct at every moment

The page set is collected first, then rows are written and charged one at a time. If your Max total charge cap is reached part-way through, the run stops there and you keep and pay for exactly the rows already written, never for a row you did not receive. A query that matches nothing writes nothing and costs nothing.

⚠️ Error reference

Derived from this Actor's own exception classes and the HTTP branches in its client. The run always reports what happened in its status message, so you never need the log to know what you got.

Raised	When	What the run does
SourceBlocked	Raised when Wikipedia refuses the request (HTTP 403 or 429 after every retry).	Run fails with: Wikipedia API blocked request (
BadQuery	Raised when the MediaWiki API rejects the query itself and returns an error object.	Run fails with: Wikipedia rejected this query (

HTTP statuses the client actually handles

Status	Handling
403	Retried with backoff. The run continues if a later attempt succeeds.
429	Retried with backoff. The run continues if a later attempt succeeds.

What each outcome costs

Outcome	Rows	Charged
Run finished	Everything the source returned for your filters	One article-returned per row
Nothing matched your filters	0	Nothing
Source refused or was unreachable	Whatever was already collected	Only the rows you got
Budget cap reached mid-run	The rows collected before the cap	Only those rows

Status messages this Actor can end on, verbatim from its own source:

• Error fetching from Wikipedia:
• Stopped at budget cap after
• No Wikipedia articles matched search criteria. Nothing charged.

❓ FAQ

Which Wikipedia languages are supported?

All public MediaWiki editions are supported, including English (en), German (de), French (fr), Spanish (es), Japanese (ja), Chinese (zh), Arabic (ar), and over 300 others.

Is HTML included in extracts?

By default, plain text introductory extracts (explaintext=1) are extracted to optimize token usage for RAG and LLM applications.

How are rate limits handled?

The scraper uses HTTP-only queries with automatic backoff and retry handling to comply with MediaWiki API guidelines.

Do I need an API key, an account or a proxy?

No. The Actor reads the source's public interface over plain HTTP, with no credential of yours anywhere in the run and no headless browser. You need an Apify account and nothing else.

What does a run cost?

One article-returned event per row written, $0.001 at list price, falling to $0.0005 on the highest Apify plan tier. Apify platform usage is absorbed rather than passed on, so the row price is the whole bill. A run that matches nothing is charged nothing.

How many rows can one run return?

Up to 500, set by the maximum-results field, which is also your cost cap. Raise it for a bigger slice, or set a Max total charge on the run and the Actor will stop cleanly when it gets there.

Can I get just some of the columns?

Yes. The dataset ships named views (overview, content, provenance); add ?view=<name> to the dataset items URL, or pick one in the Output tab, and you get that view's columns in JSON, CSV or Excel.

Can an agent call this?

Yes, through Apify's MCP server at https://mcp.apify.com with nothing to integrate on our side: search-actors finds it, fetch-actor-details returns the input schema above, and call-actor runs it.

Will the dataset still be there next month?

Yes. The run names its own default dataset at startup, and Apify keeps named storages past the retention window that deletes unnamed ones, so the run id in a row you cited a year ago still resolves.

Is the data complete?

It is exactly what the source returned for your filters. Nothing is inferred, modelled or filled in, and a field the source did not return is empty rather than guessed. The run's status message tells you what you got, so you never have to read the log to find out.

💬 Your feedback

Found a bug, or need a field this does not return yet? Open an issue on the Actor's Issues tab, or write to reapxdev@proton.me. Bugs get fixed and reasonable field requests get added.

---

Unofficial - not affiliated with Wikipedia / Wikimedia Foundation. Collects public data only. reapx. Contact reapxdev@proton.me.