# Wikipedia Scraper - Articles & Content for AI / RAG (`flash_scraper/wikipedia-scraper`) Actor Search Wikipedia in any language and get clean article rows via the free MediaWiki API: title, URL, intro summary or full plaintext content, word count, categories, thumbnail & last-edited date. Perfect for RAG, LLM training data & research. No API key. Export CSV/JSON/Excel. - **URL**: https://apify.com/flash\_scraper/wikipedia-scraper.md - **Developed by:** [Flash Scrape](https://apify.com/flash_scraper) (community) - **Categories:** AI, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $3.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Wikipedia Scraper - Articles & Content for AI / RAG **This Wikipedia scraper turns the world's largest encyclopedia into clean, structured data rows** — search by keyword or fetch exact article titles, in **any of Wikipedia's 300+ language editions**, and get one flat row per article: title, URL, an intro **summary** or the **full plaintext content**, word count, categories, thumbnail, and last-edited date. No HTML soup, no wiki markup, no `[edit]` links — just clean text ready to drop into a vector store, a fine-tuning set, or a spreadsheet. Built for **AI / RAG pipelines, LLM training datasets, researchers, and content teams** who need Wikipedia content as data — straight from the official MediaWiki API, no API key, no anti-bot, no proxies. --- ### Why use it - **Two modes: search or exact titles** — cast a wide net with a `searchQuery` ("artificial intelligence" returns the top matching articles), or pass a list of exact `titles` you already know you want. Both in one actor. - **Summary or full content** — flip `fullContent` off for a tight intro summary (great for cards, previews, embeddings of short chunks), or on for the entire article as clean plaintext (great for chunking into a RAG store or a training corpus). - **Clean plaintext, not HTML** — the MediaWiki `explaintext` extract strips markup, tables, and reference clutter, so you get readable prose with no post-processing. - **Any language edition** — set `language` to `en`, `es`, `fr`, `de`, `ar`, `zh`, or any of Wikipedia's 300+ editions to build multilingual datasets. - **Rich metadata per row** — `word_count`, up to 10 `categories`, a `thumbnail_url`, `last_edited` timestamp, and the canonical article `url` for attribution. - **Batched + resilient** — content is fetched in batches with retry/backoff; if Wikipedia is briefly unavailable the run exits cleanly with a status message and you're not charged for an empty run. - **No API key, no proxies** — the MediaWiki API is free and keyless. It runs fine on datacenter IPs. --- ### What it does You give it a search term (or a list of titles) and a language. The actor queries the MediaWiki search API to find the top matching articles, then pulls each article's content and metadata in batches of 20. Every article is flattened into one clean row and pushed to the dataset. Export to CSV, JSON, or Excel, or pull it straight into your pipeline via the API. ### Use cases - **RAG / retrieval systems** — build a knowledge base for a chatbot or agent. Pull full-content articles on a topic, chunk them, embed them, and ground your LLM in factual, citable sources with a real URL for each. - **LLM training / fine-tuning data** — assemble a clean, deduplicated text corpus on any domain or in any language without scraping raw HTML or parsing wiki markup. - **Research & literature scans** — grab the top 100 articles on a subject and get a word-count-ranked overview with summaries and categories in minutes. - **Content aggregation & enrichment** — pull authoritative summaries, thumbnails, and categories to enrich a product catalog, a glossary, a news app, or an internal wiki. - **Multilingual datasets** — run the same query across `en`, `es`, `fr`, and more to build parallel or comparative corpora. --- ### How to use it 1. Enter a **search query** (e.g. `artificial intelligence`) — or leave it and provide a list of exact **titles** instead. 2. Set the **language** edition (default `en`). 3. Turn **full article content** on for the whole article text, or leave it off for the intro summary. 4. Set **max articles** and run → get a clean, deduped article dataset. #### Input | Field | Type | Description | |---|---|---| | `searchQuery` | string | A search term; returns the top matching articles. Use this **or** `titles`. | | `titles` | array | A list of exact article titles to fetch (e.g. `["Alan Turing", "Python (programming language)"]`). | | `language` | string | Wikipedia language edition code (`en`, `es`, `fr`, `de`, `ar`, `zh`, …). Default `en`. | | `fullContent` | boolean | `false` (default) returns an intro `summary`; `true` returns the full `content`. | | `maxItems` | integer | Max articles to return (1–500). Default `25`. | You must provide **at least one** of `searchQuery` or `titles`. **Example input:** ```json { "searchQuery": "artificial intelligence", "language": "en", "maxItems": 25, "fullContent": false } ```` #### Output fields | Field | Description | |---|---| | `title` | Article title | | `pageid` | Wikipedia page ID (stable identifier) | | `url` | Canonical article URL (use for attribution) | | `summary` | Intro extract — present when `fullContent` is `false` | | `content` | Full article plaintext — present when `fullContent` is `true` | | `word_count` | Word count of the returned text | | `categories` | Up to 10 article categories | | `thumbnail_url` | Lead image thumbnail URL (if the article has one) | | `last_edited` | Timestamp of the last edit (`touched`) | | `language` | Language edition of the article | #### JSON output sample ```json { "title": "Artificial intelligence", "pageid": 1164, "url": "https://en.wikipedia.org/wiki/Artificial_intelligence", "summary": "Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making...", "word_count": 142, "categories": [ "Category:Artificial intelligence", "Category:Cybernetics", "Category:Computational fields of study" ], "thumbnail_url": null, "last_edited": "2026-07-07T06:31:50Z", "language": "en" } ``` Results render as a clean, sortable table on the Output tab and export to CSV, JSON, or Excel. #### Example output A real sample from a live run (`searchQuery: "artificial intelligence"`, `en`): | Title | pageid | Words | URL | |---|---|---|---| | Artificial intelligence | 1164 | 142 | https://en.wikipedia.org/wiki/Artificial\_intelligence | | Artificial general intelligence | 586357 | 118 | https://en.wikipedia.org/wiki/Artificial\_general\_intelligence | | A.I. Artificial Intelligence | 142224 | 96 | https://en.wikipedia.org/wiki/A.I.\_Artificial\_Intelligence | *** ### Use with AI agents & automation Run from the Apify **MCP** server so AI agents (Claude, ChatGPT, Cursor) can pull Wikipedia articles as a tool call, schedule runs via **Make**, **n8n**, or **Zapier** to feed a vector database, or sync the dataset to **Google Sheets**. Clean flat JSON with a stable `pageid` and a source `url` drops into RAG and dataset pipelines with no glue code. *** ### Pricing **Pay-per-event — charged per article delivered.** The source data is the free public MediaWiki API (no proxy or third-party cost), so you only pay for the clean, structured rows you actually receive — no subscription, no charge for empty runs. New Apify accounts get free platform credits each month, so you can pull and test a full topic before paying anything. See the Apify Store page for the current per-result price. *** ### FAQ **Where does the data come from?** The official **MediaWiki API** that powers Wikipedia — the same API Wikipedia's own apps use. It's public, keyless, and returns official article content. **Do I need an API key?** No. It works key-free, with no proxies. **Is this the full article or just the intro?** Your choice. Leave `fullContent` off for the intro summary; turn it on for the entire article as clean plaintext. **What about licensing and attribution?** Wikipedia article text is released under **CC BY-SA** (Creative Commons Attribution-ShareAlike). You're free to reuse it — including for AI/RAG and commercial use — as long as you **attribute Wikipedia and share adaptations under the same license**. Each row includes the article `url` so you can attribute the source. Review the [Wikipedia reuse guidelines](https://en.wikipedia.org/wiki/Wikipedia:Reusing_Wikipedia_content) for your specific use. **Which languages are supported?** All of Wikipedia's 300+ editions — just set the `language` code (`en`, `es`, `fr`, `de`, `ja`, `ar`, `zh`, …). **Why did an exact title return nothing?** Titles are matched exactly (capitalization is auto-corrected, but spelling and disambiguation like `(programming language)` must match). If unsure, use a `searchQuery` instead and let the search find the article. **Can I export to CSV or Google Sheets?** Yes — CSV, JSON, or Excel from the Output tab, or sync to Google Sheets via Make, n8n, or Zapier. *** ### Other Flash Scrape scrapers - [AI Web Scraper](https://apify.com/flash_scraper/ai-web-scraper) — extract structured data from any web page with AI - [Google SERP Scraper](https://apify.com/flash_scraper/google-serp-scraper) — Google search results as clean rows Questions or a field you need added? Reach out via the **Issues** tab on the Actor's Apify Store page — happy to help. # Actor input Schema ## `searchQuery` (type: `string`): A search term — the actor returns the top matching Wikipedia articles, e.g. 'artificial intelligence', 'climate change', 'quantum computing'. Use this OR the exact titles list below. ## `titles` (type: `array`): A list of exact Wikipedia article titles to fetch, e.g. \['Python (programming language)', 'Alan Turing']. Titles are matched exactly (Wikipedia auto-corrects capitalization). Use this instead of a search query when you already know which articles you want. ## `language` (type: `string`): Wikipedia language edition code, e.g. 'en' (English), 'es' (Spanish), 'fr' (French), 'de' (German), 'ar' (Arabic), 'zh' (Chinese). ## `fullContent` (type: `boolean`): Off (default) returns the intro summary in a `summary` field. On returns the full article plaintext in a `content` field — ideal for RAG / LLM training datasets. ## `maxItems` (type: `integer`): Maximum number of articles to return. ## Actor input object example ```json { "searchQuery": "artificial intelligence", "titles": [], "language": "en", "fullContent": false, "maxItems": 25 } ``` # Actor output Schema ## `results` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchQuery": "artificial intelligence", "titles": [], "language": "en", "maxItems": 25 }; // Run the Actor and wait for it to finish const run = await client.actor("flash_scraper/wikipedia-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchQuery": "artificial intelligence", "titles": [], "language": "en", "maxItems": 25, } # Run the Actor and wait for it to finish run = client.actor("flash_scraper/wikipedia-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchQuery": "artificial intelligence", "titles": [], "language": "en", "maxItems": 25 }' | apify call flash_scraper/wikipedia-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=flash_scraper/wikipedia-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Wikipedia Scraper - Articles & Content for AI / RAG", "description": "Search Wikipedia in any language and get clean article rows via the free MediaWiki API: title, URL, intro summary or full plaintext content, word count, categories, thumbnail & last-edited date. Perfect for RAG, LLM training data & research. No API key. Export CSV/JSON/Excel.", "version": "0.1", "x-build-id": "TzBkaRZEr5zqe1uvh" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/flash_scraper~wikipedia-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-flash_scraper-wikipedia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/flash_scraper~wikipedia-scraper/runs": { "post": { "operationId": "runs-sync-flash_scraper-wikipedia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/flash_scraper~wikipedia-scraper/run-sync": { "post": { "operationId": "run-sync-flash_scraper-wikipedia-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "searchQuery": { "title": "Search query", "type": "string", "description": "A search term — the actor returns the top matching Wikipedia articles, e.g. 'artificial intelligence', 'climate change', 'quantum computing'. Use this OR the exact titles list below." }, "titles": { "title": "Exact article titles", "type": "array", "description": "A list of exact Wikipedia article titles to fetch, e.g. ['Python (programming language)', 'Alan Turing']. Titles are matched exactly (Wikipedia auto-corrects capitalization). Use this instead of a search query when you already know which articles you want.", "items": { "type": "string" } }, "language": { "title": "Language edition", "type": "string", "description": "Wikipedia language edition code, e.g. 'en' (English), 'es' (Spanish), 'fr' (French), 'de' (German), 'ar' (Arabic), 'zh' (Chinese).", "default": "en" }, "fullContent": { "title": "Full article content", "type": "boolean", "description": "Off (default) returns the intro summary in a `summary` field. On returns the full article plaintext in a `content` field — ideal for RAG / LLM training datasets.", "default": false }, "maxItems": { "title": "Max articles", "minimum": 1, "maximum": 500, "type": "integer", "description": "Maximum number of articles to return.", "default": 25 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```