# Wikipedia Scraper (`anyxsolutions/wikipedia-scraper`) Actor Scrape Wikipedia articles in any language by search term, category or link: full plain text split into sections, lead summary, short description, categories, coordinates, images, Wikidata ID, word count, last edit and optional 30-day pageviews. Clean JSON, no API key, no proxy. - **URL**: https://apify.com/anyxsolutions/wikipedia-scraper.md - **Developed by:** [Anyx Solutions](https://apify.com/anyxsolutions) (community) - **Categories:** AI, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $2.50 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ![banner](https://anyx.solutions/banners/wikipedia-scraper.png) ## Wikipedia Scraper **Wikipedia articles in any language β€” by search, category or link β€” as clean text with sections, metadata and pageviews.** Wikipedia is the largest reference work ever written: more than 60 million articles across over 300 language editions, each with a short description, categories, a Wikidata ID and, for places, coordinates. This Actor turns it into a dataset: paste article, category or search links from any language edition, or type your own search terms, and get every matching article with its full plain text split into sections with their headings, the lead summary, short description, categories, lead image, coordinates, Wikidata ID, word count, last edit date and revision, how many other languages cover it, and optionally how many people read it over the last 30 days. Formulas come out as inline LaTeX, not pages of stray symbols. There is no login, no API key, and no proxy to configure. Built for RAG and LLM knowledge bases, training and evaluation corpora, entity enrichment, content and SEO research, and trend tracking. ### ⚑ Quick start ```json { "startUrls": [ { "url": "https://en.wikipedia.org/w/index.php?search=quantum+computing&title=Special%3ASearch&fulltext=1&ns0=1" } ], "maxItems": 5 } ``` Or every article in a category, with 30 days of pageviews: ```json { "startUrls": [ { "url": "https://en.wikipedia.org/wiki/Category:Machine_learning_algorithms" } ], "includePageviews": true, "maxItems": 100 } ``` Or search terms in another language: ```json { "searchTerms": ["Energiewende", "Ada Lovelace"], "language": "de", "maxItems": 20 } ``` ### 🧩 Input | Field | Type | Required | Default | Description | | -------------------- | ------- | -------- | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `startUrls` | array | no | β€” | Wikipedia URLs in any language: articles (`en.wikipedia.org/wiki/`, mobile `en.m.wikipedia.org` links too), categories (`/wiki/Category:<Name>`, returns its articles) or search results pages (`/w/index.php?search=…`). | | `searchTerms` | array | no | β€” | Search terms, one search per line. Each returns the best-matching articles in `language`. | | `language` | string | no | `"en"` | Wikipedia language edition for the search terms, as its code: `en`, `de`, `fr`, `es`, `ja`, `zh`, … Start URLs keep their own language. | | `maxItems` | integer | no | β€” | Maximum number of articles per start URL or search term. The input form starts at 5; leave it empty to get every match (Wikipedia caps one search at 10,000 articles). | | `includeContent` | boolean | no | `true` | Return the whole article as plain text and as sections. Turn it off to keep only the summary and metadata. | | `includePageviews` | boolean | no | `false` | Add views by people over the last 30 days, in total and per day. | | `proxyConfiguration` | object | no | `{ "useApifyProxy": false }` | Proxy settings. Wikipedia needs no proxy. | Add at least one start URL or one search term. A category returns the articles filed directly in it; sub-categories are not followed, so add them as their own start URLs when you want them. Search results come in Wikipedia's relevance order. ### πŸ“€ Output Each article is one dataset row. #### Fields | Field | Type | Description | | ---------------- | ------ | ----------------------------------------------------------------------------------------------------------------------- | | `pageId` | number | Wikipedia page ID, unique within one language edition. | | `title` | string | Article title. Redirects are followed, so this is the article's current title. | | `url` | string | Link to the article. | | `language` | string | Language edition, such as `en` or `de`. | | `description` | string | The short description shown under the title in search and apps, such as "Tower in Paris, France". | | `summary` | string | The lead section: the article's introduction, as plain text. | | `contentText` | string | The whole article as plain text, each section's heading on its own line; `null` when `includeContent` is off. | | `sections` | array | The article's sections in order, each with `heading`, `level` (2 for a main section, 3 for a subsection, …) and `text`. | | `wordCount` | number | Words in the article text. | | `lastModified` | string | When the article was last edited (ISO 8601). | | `revisionId` | number | ID of the latest revision, for citing or diffing an exact version. | | `categories` | array | The article's visible categories, without the `Category:` prefix. Maintenance categories are left out. | | `thumbnailUrl` | string | The article's lead image, about 640 pixels wide. | | `imageUrl` | string | The lead image at full size. | | `coordinates` | object | `lat` and `lon` of the article's subject, for places and buildings; `null` otherwise. | | `wikidataId` | string | The Wikidata item, such as `Q243`, which links the article to its versions in every language and to structured data. | | `languagesCount` | number | How many other Wikipedia language editions have an article on the subject. | | `pageviews30d` | number | Views by people (not bots) over the last 30 days; `null` unless `includePageviews` is on. | | `pageviewsDaily` | array | The same views per day, each with `date` and `views`; `null` unless `includePageviews` is on. | | `searchQuery` | string | The search term that found the article; `null` for articles from a link or a category. | <details><summary>Example output</summary> ```json { "pageId": 9232, "title": "Eiffel Tower", "url": "https://en.wikipedia.org/wiki/Eiffel_Tower", "language": "en", "description": "Tower in Paris, France", "summary": "The Eiffel Tower (EYE-fΙ™l; French: Tour Eiffel [tuʁ Ι›fΙ›l]) is a lattice tower on the Champ de Mars in Paris, France. It is named after the engineer Gustave Eiffel, whose company designed and built the tower from 1887 to 1889. …", "contentText": "The Eiffel Tower (EYE-fΙ™l; French: Tour Eiffel [tuʁ Ι›fΙ›l]) is a lattice tower on the Champ de Mars in Paris, France. It …", "sections": [ { "heading": "History", "level": 2, "text": null }, { "heading": "Origin", "level": 3, "text": "The design of the Eiffel Tower is attributed to Maurice Koechlin and Γ‰mile Nouguier, two senior engineers working for …" } ], "wordCount": 7580, "lastModified": "2026-09-25T19:49:48Z", "revisionId": 1376706466, "categories": [ "1889 establishments in France", "Architectural controversies", "Buildings and structures in the 7th arrondissement of Paris", "Eiffel Tower" ], "thumbnailUrl": "https://thumb.wikimedia.org/wikipedia/commons/thumb/8/85/Tour_Eiffel_Wikimedia_Commons_%28cropped%29.jpg/960px-Tour_Eiffel_Wikimedia_Commons_%28cropped%29.jpg", "imageUrl": "https://upload.wikimedia.org/wikipedia/commons/8/85/Tour_Eiffel_Wikimedia_Commons_%28cropped%29.jpg", "coordinates": { "lat": 48.85822222, "lon": 2.2945 }, "wikidataId": "Q243", "languagesCount": 175, "pageviews30d": 124575, "pageviewsDaily": [ { "date": "2026-09-13", "views": 3577 }, { "date": "2026-09-14", "views": 4240 } ], "searchQuery": null } ``` </details> ### πŸ’‘ Use cases - **RAG and LLM knowledge bases** β€” clean, sectioned article text ready to chunk by heading and embed. - **Training and evaluation corpora** β€” pull a whole topic or category in any language, with revision IDs for reproducibility. - **Entity enrichment** β€” add descriptions, categories, coordinates, images and Wikidata IDs to lists of companies, people or places. - **Content and SEO research** β€” see what Wikipedia covers on a topic, how long the articles are and when they were last edited. - **Trend and attention tracking** β€” compare 30-day pageviews across articles to see what people are reading. ### ❓ FAQ - **Do I need an API key or a Wikipedia account?** No. The Actor reads Wikipedia's public articles. - **Do I need a proxy?** No. It runs without one by default. - **Which languages work?** Every Wikipedia language edition. Paste a link from that edition, or set `language` to its code (the part before `.wikipedia.org`, such as `pt` or `zh-yue`). - **Does a category include its sub-categories?** No, only the articles filed directly in it. Add a sub-category as its own start URL to include it. - **Why are tables, infoboxes and reference numbers missing from the text?** The text is the article's prose, as a reader would quote it. Infoboxes and tables are left out; the structured facts they hold are on the article's Wikidata item, linked by `wikidataId`. - **How are formulas returned?** As inline LaTeX, for example `F_{T}(x)=\sum _{t=1}^{T}f_{t}(x)`. - **Why are `coordinates`, `thumbnailUrl` or `description` empty?** Not every article has them: only places carry coordinates, some articles have no image, and smaller language editions have fewer short descriptions. - **Why does `pageviewsDaily` have 29 days instead of 30?** Wikipedia publishes each day's views a few hours after midnight UTC; a run early in the day gets the 29 days published so far. - **Is `wordCount` right for Chinese or Japanese?** Yes. It uses each language's word boundaries, so text written without spaces is counted by words, not by lines. - **Can I reuse the article text?** Yes, under its licence. Wikipedia's text is licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/): attribute Wikipedia and the article (every row carries the article's `url` for the link), and share anything you build from the text under the same ShareAlike licence. Images, including `thumbnailUrl`, have their own licences, shown on each file's page. See [Reusing Wikipedia content](https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_content). ### πŸ”— More scrapers by Anyx - [Web Content Scraper](https://apify.com/anyxsolutions/web-content-scraper) - [Google Search Scraper](https://apify.com/anyxsolutions/google-search-scraper) - [arXiv Scraper](https://apify.com/anyxsolutions/arxiv-scraper) - [OpenAlex Scraper](https://apify.com/anyxsolutions/openalex-scraper) ### 🀝 Anyx Solutions We build custom scrapers and data-extraction pipelines. - Email: tantosthor@gmail.com ### πŸ–Ό Image credit Image credit: [wikipedia.org](https://www.wikipedia.org/) # Actor input Schema ## `startUrls` (type: `array`): Wikipedia pages in any language: an article such as https://en.wikipedia.org/wiki/Eiffel_Tower (mobile links work too), a category such as https://en.wikipedia.org/wiki/Category:Machine_learning_algorithms (returns its articles), or a search results page such as https://en.wikipedia.org/w/index.php?search=quantum+computing. ## `searchTerms` (type: `array`): Words to search Wikipedia for, one search per line, for example renewable energy or "Ada Lovelace". Each term returns the best-matching articles in the language below. ## `language` (type: `string`): Wikipedia language edition for the search terms, as its code, for example en (English), de (German), fr (French), es (Spanish), ja (Japanese) or zh (Chinese). Start URLs keep their own language. ## `maxItems` (type: `integer`): Maximum number of articles to return per start URL or search term. Leave empty to get every match (Wikipedia caps one search at 10,000 articles). ## `includeContent` (type: `boolean`): Return the whole article as plain text and as sections with their headings. Turn it off to keep only the summary and metadata, for smaller datasets. ## `includePageviews` (type: `boolean`): Add how many times people viewed each article over the last 30 days, in total and per day. Takes one extra request per article. ## `proxyConfiguration` (type: `object`): Proxy settings. Wikipedia needs no proxy; the Actor keeps to Wikimedia's polite request rate. ## Actor input object example ```json { "startUrls": [ { "url": "https://en.wikipedia.org/w/index.php?search=quantum+computing&title=Special%3ASearch&fulltext=1&ns0=1" } ], "language": "en", "maxItems": 5, "includeContent": true, "includePageviews": false, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '<YOUR_API_TOKEN>' with your token const client = new ApifyClient({ token: '<YOUR_API_TOKEN>', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://en.wikipedia.org/w/index.php?search=quantum+computing&title=Special%3ASearch&fulltext=1&ns0=1" } ], "maxItems": 5 }; // Run the Actor and wait for it to finish const run = await client.actor("anyxsolutions/wikipedia-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '<YOUR_API_TOKEN>' with your token. client = ApifyClient("<YOUR_API_TOKEN>") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://en.wikipedia.org/w/index.php?search=quantum+computing&title=Special%3ASearch&fulltext=1&ns0=1" }], "maxItems": 5, } # Run the Actor and wait for it to finish run = client.actor("anyxsolutions/wikipedia-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://en.wikipedia.org/w/index.php?search=quantum+computing&title=Special%3ASearch&fulltext=1&ns0=1" } ], "maxItems": 5 }' | apify call anyxsolutions/wikipedia-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,anyxsolutions/wikipedia-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/CWsidpahBl1Zi7kHy/builds/cy5LuES36dIAPwBht/openapi.json