Wikipedia Scraper

Wikipedia articles in any language — by search, category or link — as clean text with sections, metadata and pageviews.

Wikipedia is the largest reference work ever written: more than 60 million articles across over 300 language editions, each with a short description, categories, a Wikidata ID and, for places, coordinates. This Actor turns it into a dataset: paste article, category or search links from any language edition, or type your own search terms, and get every matching article with its full plain text split into sections with their headings, the lead summary, short description, categories, lead image, coordinates, Wikidata ID, word count, last edit date and revision, how many other languages cover it, and optionally how many people read it over the last 30 days. Formulas come out as inline LaTeX, not pages of stray symbols. There is no login, no API key, and no proxy to configure. Built for RAG and LLM knowledge bases, training and evaluation corpora, entity enrichment, content and SEO research, and trend tracking.

⚡ Quick start

{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/w/index.php?search=quantum+computing&title=Special%3ASearch&fulltext=1&ns0=1"
    }
  ],
  "maxItems": 5
}

Or every article in a category, with 30 days of pageviews:

{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Category:Machine_learning_algorithms"
    }
  ],
  "includePageviews": true,
  "maxItems": 100
}

Or search terms in another language:

{
  "searchTerms": ["Energiewende", "Ada Lovelace"],
  "language": "de",
  "maxItems": 20
}

🧩 Input

Field	Type	Required	Default	Description
startUrls	array	no	—	Wikipedia URLs in any language: articles (en.wikipedia.org/wiki/<Title>, mobile en.m.wikipedia.org links too), categories (/wiki/Category:<Name>, returns its articles) or search results pages (/w/index.php?search=…).
searchTerms	array	no	—	Search terms, one search per line. Each returns the best-matching articles in language.
language	string	no	"en"	Wikipedia language edition for the search terms, as its code: en, de, fr, es, ja, zh, … Start URLs keep their own language.
maxItems	integer	no	—	Maximum number of articles per start URL or search term. The input form starts at 5; leave it empty to get every match (Wikipedia caps one search at 10,000 articles).
includeContent	boolean	no	true	Return the whole article as plain text and as sections. Turn it off to keep only the summary and metadata.
includePageviews	boolean	no	false	Add views by people over the last 30 days, in total and per day.
proxyConfiguration	object	no	{ "useApifyProxy": false }	Proxy settings. Wikipedia needs no proxy.

Add at least one start URL or one search term. A category returns the articles filed directly in it; sub-categories are not followed, so add them as their own start URLs when you want them. Search results come in Wikipedia's relevance order.

📤 Output

Each article is one dataset row.

Fields

Field	Type	Description
pageId	number	Wikipedia page ID, unique within one language edition.
title	string	Article title. Redirects are followed, so this is the article's current title.
url	string	Link to the article.
language	string	Language edition, such as en or de.
description	string	The short description shown under the title in search and apps, such as "Tower in Paris, France".
summary	string	The lead section: the article's introduction, as plain text.
contentText	string	The whole article as plain text, each section's heading on its own line; null when includeContent is off.
sections	array	The article's sections in order, each with heading, level (2 for a main section, 3 for a subsection, …) and text.
wordCount	number	Words in the article text.
lastModified	string	When the article was last edited (ISO 8601).
revisionId	number	ID of the latest revision, for citing or diffing an exact version.
categories	array	The article's visible categories, without the Category: prefix. Maintenance categories are left out.
thumbnailUrl	string	The article's lead image, about 640 pixels wide.
imageUrl	string	The lead image at full size.
coordinates	object	lat and lon of the article's subject, for places and buildings; null otherwise.
wikidataId	string	The Wikidata item, such as Q243, which links the article to its versions in every language and to structured data.
languagesCount	number	How many other Wikipedia language editions have an article on the subject.
pageviews30d	number	Views by people (not bots) over the last 30 days; null unless includePageviews is on.
pageviewsDaily	array	The same views per day, each with date and views; null unless includePageviews is on.
searchQuery	string	The search term that found the article; null for articles from a link or a category.

💡 Use cases

• RAG and LLM knowledge bases — clean, sectioned article text ready to chunk by heading and embed.
• Training and evaluation corpora — pull a whole topic or category in any language, with revision IDs for reproducibility.
• Entity enrichment — add descriptions, categories, coordinates, images and Wikidata IDs to lists of companies, people or places.
• Content and SEO research — see what Wikipedia covers on a topic, how long the articles are and when they were last edited.
• Trend and attention tracking — compare 30-day pageviews across articles to see what people are reading.

❓ FAQ

• Do I need an API key or a Wikipedia account? No. The Actor reads Wikipedia's public articles.
• Do I need a proxy? No. It runs without one by default.
• Which languages work? Every Wikipedia language edition. Paste a link from that edition, or set language to its code (the part before .wikipedia.org, such as pt or zh-yue).
• Does a category include its sub-categories? No, only the articles filed directly in it. Add a sub-category as its own start URL to include it.
• Why are tables, infoboxes and reference numbers missing from the text? The text is the article's prose, as a reader would quote it. Infoboxes and tables are left out; the structured facts they hold are on the article's Wikidata item, linked by wikidataId.
• How are formulas returned? As inline LaTeX, for example F_{T}(x)=\sum _{t=1}^{T}f_{t}(x).
• Why are coordinates, thumbnailUrl or description empty? Not every article has them: only places carry coordinates, some articles have no image, and smaller language editions have fewer short descriptions.
• Why does pageviewsDaily have 29 days instead of 30? Wikipedia publishes each day's views a few hours after midnight UTC; a run early in the day gets the 29 days published so far.
• Is wordCount right for Chinese or Japanese? Yes. It uses each language's word boundaries, so text written without spaces is counted by words, not by lines.
• Can I reuse the article text? Yes, under its licence. Wikipedia's text is licensed under CC BY-SA 4.0: attribute Wikipedia and the article (every row carries the article's url for the link), and share anything you build from the text under the same ShareAlike licence. Images, including thumbnailUrl, have their own licences, shown on each file's page. See Reusing Wikipedia content.

🔗 More scrapers by Anyx

• Web Content Scraper
• Google Search Scraper
• arXiv Scraper
• OpenAlex Scraper

🤝 Anyx Solutions

We build custom scrapers and data-extraction pipelines.

• Email: tantosthor@gmail.com

🖼 Image credit

Image credit: wikipedia.org